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VALIDATION  OF  THE  NONCOMMISSIONED  OFFICER  SPECIAL  ASSIGNMENT 
BATTERY 

EXECUTIVE  SUMMARY 


Research  Requirement: 

The  Department  of  the  Army  (DA)  and  the  United  States  Army  Recruiting  Command 
(USAREC)  must  recruit  large  numbers  of  personnel  annually  that  meet  or  exceed  enlistment 
standards  that  many  civilian  organizations  do  not  require.  In  FY12,  the  combined  Active  Army 
and  Army  Reserve  accession  mission  was  over  74,000  personnel,  which  is  somewhat  lower  than 
historical  levels.  The  Army  has  more  than  8,000  Active  Army  and  Army  Reserve  Soldiers 
assigned  as  recruiters  in  more  than  1,600  recruiting  stations  throughout  the  U.S.  and  overseas 
(USAREC,  2012).  To  fulfill  their  annual  accession  mission,  USAREC  requires  a  qualified  and 
dedicated  team  of  Soldiers  that  can  be  trained  to  become  effective  recruiters. 

The  U.S.  Army  Research  Institute  for  the  Behavioral  and  Social  Sciences  (ARI) 
conducted  a  series  of  research  activities  from  2000  to  2005  to  develop  and  validate  the 
Noncommissioned  Officer  Leadership  Skills  Inventory  (NLSI)  (Borman  et  ah,  2004;  White, 
Borman,  &  Bowles,  2001).  This  research  was  intended  to  help  the  Army  move  forward  with  its 
efforts  to  develop  and  implement  a  screening  process  for  selecting  Army  recruiters.  The  NLSI 
measures  skills  and  abilities  related  to  recruiter  performance  such  as  work  orientation, 
interpersonal  skills,  and  leadership  capability.  Predictive  and  concurrent  validation  research 
results  demonstrated  that  the  NLSI  was  related  to  attrition  from  recruiter  training,  recruiter 
performance,  and  production  (recruits  enlisted)  (Horgen  et  al.,  2006;  White  et  al.,  2002;  White  et 
ah,  2004).  Additional  research  was  conducted  to  improve  the  NLSI  prediction  model  using  a 
statistical  learning  and  data  mining  method.  A  revised  prediction  algorithm  was  created  to  select 
Soldiers  for  assignment  to  recruiting  duty  (Halstead,  2009). 

In  2008,  the  Deputy  Chief  of  Staff  (DCS),  G-3,  and  G-l  collectively  implemented  the 
NLSI,  later  named  the  Warrior  Attributes  Inventory  (WAI),  to  select  Soldiers  for  assignment  to 
Recruiting  School.  The  WAI  was  administered  at  Digital  Training  Facilities  (DTFs)  worldwide 
for  operational  testing.  However,  the  volume  of  Soldiers  testing  at  the  DTFs  was  insufficient  to 
support  full  implementation  of  a  recruiter  screening  program.  To  streamline  and  improve  the 
program,  the  U.S.  Army  Training  and  Doctrine  Command  (TRADOC)  and  Human  Resources 
Command  (HRC)  recently  requested  that  ARI  assist  with:  1)  creating  a  measure  for  unproctored, 
online  administration,  and  2)  validating  the  assessment  for  in-service  testing. 

The  primary  objective  of  the  current  research  was  to  assist  USAREC  in  developing  a  new 
instrument  that  can  be  administered  online,  in  an  unproctored  environment,  and  will  identify 
Soldiers  with  high  potential  for  recruiting  duty  success.  Our  goal  was  to  explore  the  potential 
validity  of  this  new  instrument,  the  Noncommissioned  Officer  Special  Assignment  Battery 
(NSAB). 
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Procedure: 


This  report  describes  the  two  approaches  that  were  used  to  evaluate  the  NSAB  in 
Noncommissioned  Officer  (NCO)  and  recruiter  samples.  First,  the  construct  validity  of  the 
NSAB  was  examined  by  comparing  results  from  the  NSAB  and  the  WAI  in  a  sample  of  NCOs. 
Next,  the  relationship  of  the  NSAB  to  perfonnance  and  job  attitudes  was  explored  in  a  sample  of 
experienced  recruiters.  These  analyses  were  conducted  to  explore  the  extent  to  which  the  NSAB 
might  be  useful  for  NCO  in-service  testing  and  selection  into  the  recruiter  special  duty 
assignment. 

The  NSAB  is  based  on  the  Tailored  Adaptive  Personality  Assessment  System  (TAPAS), 
and  consists  of  126  items  measuring  18  personality  dimensions.  The  TAPAS  was  specifically 
developed  to  be  fake-resistant  and  administered  in  a  computerized  fonnat.  As  such,  this  measure 
is  uniquely  suited  for  the  online  administration  of  high-stakes  selection  tests  like  the  NSAB. 
Previous  research  conducted  on  the  WAI  provided  results  regarding  the  key  personality 
dimensions  that  predict  recruiter  performance.  With  this  in  mind,  TAPAS  scales  were  selected 
for  the  NSAB  that  measure  the  attributes  and  skills  previously  found  to  be  relevant  to  recruiter 
and  NCO  performance.  The  overlap  between  key  dimensions  on  the  NSAB  and  the  WAI  was 
examined  in  a  sample  of  NCOs  who  completed  both  measures. 

Our  next  step  was  to  examine  the  validity  of  the  NSAB  in  a  sample  of  recruiters  to  more 
directly  test  the  relationships  between  the  NSAB  scales  and  recruiter  success.  NSAB  and 
criterion  (Anny  Recruiter  Perfonnance  Rating  Scales,  Recruiting  Life  Questionnaire)  data  were 
collected  from  over  600  experienced  recruiters  and  their  peers  and  supervisors  from  August 
through  November  2011.  All  of  the  data  were  collected  through  unproctored,  web-based 
assessments.  Correlational  and  regression  analyses  were  conducted  on  the  NSAB  and  recruiter 
criterion  measures  to  estimate  the  validity  of  the  NSAB  for  predicting  recruiter  performance,  job 
fit,  commitment,  and  satisfaction. 

Finally,  in  a  sample  of  NCOs,  we  conducted  a  preliminary  investigation  of  the  validity  of 
the  NSAB  for  predicting  several  criteria  including  NCO  perfonnance,  commitment,  and 
retention.  Predictor  (WAI,  NSAB)  and  criterion  data  (Anny  Life  Questionnaire,  Perfonnance 
Rating  Scales)  were  collected  from  348  NCOs  and  313  supervisors  from  March  through 
September  2010.  We  then  conducted  correlational  analyses  on  the  WAI  and  NSAB  and 
correlational  and  regression  analyses  on  the  NSAB  and  NCO  criterion  measures. 

Findings: 

The  results  from  the  sample  of  experienced  recruiters  demonstrated  that  several  NSAB 
scales  were  significantly  related  to  a  number  of  recruiting  success  criteria.  Recruiters  scoring 
higher  on  the  NSAB  were  rated  as  higher  performers  by  their  supervisors  and  peers.  NSAB 
scores  also  significantly  predicted  recruiter  commitment,  job  fit,  job  stress,  and  satisfaction  with 
recruiting.  Recruiters  in  the  top  20%  on  the  NSAB  composite  were  more  committed  to  the  Army, 
experienced  less  job  stress,  reported  a  higher  degree  of  fit  with  recruiting,  and  perfonned  better 
than  those  in  the  lowest  scoring  20%.  Results  also  indicated  that  the  NSAB  measured  the  key 
non-cognitive  dimensions  shown  to  predict  Army  recruiter  perfonnance  in  previous  research. 
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These  results  suggest  that  the  NSAB  can  help  to  identify  and  select  high  potential  Soldiers  for 
recruiting  duty  assignments. 

The  primary  focus  in  the  NCO  data  collection  was  to  confirm  that  the  NSAB 
demonstrated  significant  overlap  with  the  key  dimensions  on  the  WAI  that  had  predicted 
recruiter  perfonnance  in  previous  research.  The  correlations  between  the  NSAB  and  the  WAI 
scales  indicated  that  the  two  measures  have  similar  content,  especially  on  the  key  scales  that 
were  related  to  recruiter  performance.  Further,  regression  analyses  demonstrated  that  NCOs 
scoring  highly  on  the  NSAB  reported  stronger  commitment  to  the  Anny,  better  fit  with  their 
MOS,  and  higher  levels  of  physical  fitness  (APFT). 

Utilization  and  Dissemination  of  Findings: 

The  NSAB  results  are  consistent  with  past  research  examining  the  predictors  of  recruiter 
success  (White  et  ah,  2001).  Further,  the  NSAB  improves  on  the  WAI  in  several  areas.  First,  the 
NSAB  captures  content  that  is  similar  to,  and  may  improve  upon,  the  WAI.  Second,  the  NSAB 
can  be  administered  in  an  online,  unproctored  environment,  which  would  simplify  NCO  test 
administration  procedures.  Third,  the  NSAB  was  developed  using  a  forced-choice  format  which 
is  more  resistant  to  faking  than  traditional  non-cognitive  measures.  Finally,  preliminary  evidence 
suggests  that  the  NSAB  can  help  to  identify  Soldiers  with  high  potential  for  recruiting  duty 
success.  Recruiters  with  higher  NSAB  scores  were  more  committed  to  the  Anny,  reported  a 
higher  degree  of  fit  with  recruiting,  experienced  less  job  stress,  and  received  higher  perfonnance 
ratings  than  those  with  lower  NSAB  scores. 

Building  on  the  promising  foundation  of  the  current  project,  future  research  is 
recommended  to  evaluate  the  validity  of  the  NSAB  in  a  longitudinal,  predictive  design,  and 
under  operational  conditions.  This  type  of  design  would  allow  the  Army  to  more  closely  estimate 
the  validity  of  an  operational  NSAB  recruiter  screening  program. 

The  content  and  features  of  the  NSAB  also  have  potential  for  NCO  selection  for  other 
assignments  such  as  drill  sergeant,  instructors,  or  special  operations,  and  the  authors  recommend 
further  testing  of  the  NSAB  for  use  as  an  in-service  selection  tool  for  other  NCO  duty 
assignments.  As  a  result  of  the  success  of  this  research,  ARI  is  pursuing  implementation  of  the 
NSAB  on  an  Anny-wide  accessible  platform  to  support  further  testing  and  evaluation  of 
expanded  applications  of  the  NSAB. 
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VALIDATION  OF  THE 

NONCOMMISSIONED  OFFICER  SPECIAL  ASSIGNMENT  BATTERY  (NSAB) 


CHAPTER  1:  INTRODUCTION 

The  Department  of  the  Army  (DA)  and  the  United  States  Anny  Recruiting  Command 
(USAREC)  must  recruit  large  numbers  of  personnel  annually  that  meet  or  exceed  enlistment 
standards  that  many  civilian  organizations  do  not  require.  In  FY12,  the  combined  Active  Anny 
and  Anny  Reserve  accession  mission  was  over  74,000  personnel,  which  is  somewhat  lower  than 
historical  levels.  The  Army  has  more  than  8,000  Active  Army  and  Army  Reserve  Soldiers 
assigned  as  recruiters  in  more  than  1,600  recruiting  stations  throughout  the  U.S.  and  overseas 
(USAREC,  2012).  To  fulfill  their  annual  accession  mission,  USAREC  requires  a  qualified  and 
dedicated  team  of  Soldiers  that  can  be  trained  to  become  effective  recruiters. 

The  U.S.  Anny  Research  Institute  for  the  Behavioral  and  Social  Sciences  (ARI) 
conducted  a  series  of  research  activities  from  2000  to  2005  to  develop  and  validate  a  measure  to 
select  Soldiers  with  high  potential  for  recruiting  duty  success.  To  begin,  ARI  conducted  an 
extensive  review  of  the  literature  on  military  recruitment  to  identify  individual  difference  and 
other  factors  likely  to  contribute  to  high  levels  of  recruiting  performance  (Bonnan,  Horgen,  & 
Penney,  2000).  Next,  ARI  developed  a  paper-and-pencil  test  battery  composed  of  many  of  these 
factors  and  conducted  preliminary  research  to  evaluate  the  empirical  validity  of  the  test  battery. 

Based  on  the  results  of  concurrent  validation  research,  ARI  developed  the 
Noncommissioned  Officer  Leadership  Skills  Inventory  (NLSI),  which  was  originally  a  paper- 
and-pencil  measure  administered  in  a  proctored  environment  (Bonnan  et  ah,  2004;  White, 
Borman,  &  Bowles,  2001).  The  NLSI  measures  skills  and  abilities  related  to  recruiter 
performance  such  as  work  orientation,  interpersonal  skills,  and  leadership  capability.  Predictive 
validation  research  conducted  with  thousands  of  Anny  recruiters  demonstrated  that  the  NLSI 
was  related  to  attrition  from  recruiter  training,  recruiter  perfonnance,  and  production  (recruits 
enlisted)  (Horgen  et  ah,  2006;  White  et  al.,  2002;  White  et  ah,  2004).  Additional  research  was 
conducted  to  improve  the  NLSI  prediction  model  using  a  statistical  learning  and  data  mining 
method.  A  revised  prediction  algorithm  was  created  to  select  Soldiers  for  assignment  to 
recruiting  duty  (Halstead,  2009). 

In  2008,  the  Deputy  Chief  of  Staff  (DCS),  G-3,  and  G-l  collectively  implemented  a 
computerized  version  of  the  NLSI,  later  named  the  Warrior  Attributes  Inventory  (WAI),  to  select 
Soldiers  for  assignment  to  Recruiting  School.  The  WAI  was  successfully  deployed  at  Digital 
Training  Facilities  (DTFs)  worldwide  for  operational  testing.  However,  the  volume  of  Soldiers 
testing  at  the  DTFs  was  insufficient  to  support  full  implementation  of  a  recruiter  screening 
program.  To  streamline  and  improve  the  program,  the  U.S.  Army  Training  and  Doctrine 
Command  (TRADOC)  and  Human  Resources  Command  (HRC)  recently  requested  that  ARI 
assist  with:  1)  creating  a  measure  for  unproctored,  online  administration,  and  2)  validating  the 
assessment  for  in-service  testing. 

In  the  current  project,  ARI,  Personnel  Decisions  Research  Institutes  (PDRI),  and 
Drasgow  Consulting  Group  (DCG)  conducted  research  to  assist  USAREC  in  developing  a  new 
instrument  that  could  be  administered  in  an  online,  unproctored  environment  to  identify  Soldiers 
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with  high  potential  for  recruiting  duty  success.  Our  goal  was  to  explore  the  potential  validity  of 
this  new  instrument  for  Noncommissioned  Officer  (NCO)  in-service  testing  and  selection  for 
recruiting  duty  assignment.  Next,  we  describe  this  new  measure,  the  Noncommissioned  Officer 
Special  Assignment  Battery  (NSAB). 

Noncommissioned  Officer  Special  Assignment  Battery  (NSAB) 

The  NSAB  consists  of  126  items  measuring  18  personality  dimensions  selected  to  cover 
constructs  measured  in  the  original  WAI.  Table  1  lists  the  descriptions  of  the  personality 
dimensions  assessed  by  the  NSAB.  Two  versions  of  the  NSAB  inventory  were  developed  to 
accommodate  different  testing  needs.  The  first  is  a  static  form  that  can  be  administered  via 
paper-and-pencil  in  settings  where  computers  with  internet  access  are  unavailable;  the  second  is 
a  computerized  adaptive  test  deployed  via  a  web  application. 

Both  versions  of  the  NSAB  were  based  on  the  Tailored  Adaptive  Personality  Assessment 
System  (TAPAS).  At  the  heart  of  the  assessment  system  is  a  trait  taxonomy  comprising  21  facets 
of  the  Big  Five  personality  factors  plus  Physical  Conditioning,  which  has  been  shown  to  be 
important  for  military  applications  (Chernyshenko,  Stark,  &  Drasgow,  2010;  Chernyshenko, 
Stark,  Drasgow,  &  Roberts,  2007).  TAPAS  tests  utilize  a  multidimensional  pairwise  preference 
(MDPP)  fonnat  that  is  designed  to  be  resistant  to  faking  in  a  way  that  is  similar  to  the  Army’s 
Assessment  of  Individual  Motivation  (AIM;  Stark,  Chernyshenko,  Drasgow,  Lee,  White,  & 
Young,  2011;  White  &  Young,  1998;  White,  Young,  Heggestad,  Stark,  Drasgow,  &  Piskator, 
2004)  inventory.  The  MDPP  format  was  chosen  because  it  provides  a  more  mathematically 
tractable  alternative  for  constructing  and  scoring  adaptive  tests  using  item  response  theory  (IRT) 
(Stark,  Chernyshenko,  &  Drasgow,  2005;  Stark,  Chernyshenko,  Drasgow,  &  White,  2012). 

Initial  predictive  and  construct-related  validity  evidence  for  TAPAS  was  collected  during 
the  U.S.  Anny’s  Expanded  Enlistment  Eligibility  Metrics  (EEEM)  research  project  from  2007- 
2009  (Knapp  &  Heffner,  2010).  The  EEEM  effort  was  conducted  in  conjunction  with  ARI’s 
Army  Class  longitudinal  validation  of  multiple  experimental  non-cognitive  predictor  measures. 

In  the  EEEM  project,  new  Soldiers  completed  a  12-dimension,  95-item  nonadaptive  (or  static) 
version  of  TAPAS,  called  TAPAS-95s.  TAPAS-95s  was  administered  as  a  paper  questionnaire 
that  included  an  infonnation  sheet  showing  respondents  a  sample  item  and  illustrating  how  to 
properly  record  their  answers  to  the  “questions”  that  followed.  Respondents  were  specifically 
instructed  to  choose  the  statement  in  each  pair  that  was  “more  like  me”  and  that  they  must  make 
a  choice  even  if  they  found  it  difficult  to  do  so.  Item  responses  were  scored  using  an  updated 
version  of  Stark’s  (2002)  computer  program  for  MDPP  trait  estimation. 

Overall,  the  TAPAS-95s  showed  evidence  of  construct  and  criterion  validity  as  well  as 
incremental  validity  over  AFQT  for  predicting  several  perfonnance  criteria.  For  example,  when 
TAPAS  trait  scores  were  added  into  a  regression  analysis  based  on  a  sample  of  several  hundred 
Soldiers,  the  multiple  correlation  increased  by  .26  for  the  prediction  of  physical  fitness,  by  .16 
for  the  prediction  of  disciplinary  incidents,  and  by  .20  for  the  prediction  of  6-month  attrition 
(Allen,  Cheng,  Putka,  Hunter,  &  White,  2010).  None  of  these  criteria  were  predicted  well  by 
AFQT  alone  (predictive  validity  estimates  were  consistently  below  .10). 
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In  May  2009,  the  U.S.  Army  also  approved  the  initial  operational  test  and  evaluation 
(IOT&E)  of  the  TAPAS  for  use  with  Anny  applicants  at  Military  Entrance  Processing  Stations 
(MEPS).  In  collaboration  with  the  Anny  Research  Institute,  DCG  developed  the  three 
computerized  forms  of  TAPAS  implemented  in  the  MEPS.  Validation  research  on  these  versions 
of  TAPAS  has  shown  that  the  TAPAS  scales  have  validity  for  predicting  Army- wide  (Nye, 
Drasgow,  Stark,  Chernyshenko,  &  White,  2012)  and  MOS-specific  criteria  (Nye,  Drasgow, 
Chernyshenko,  Stark,  Kubisiak,  White,  &  Jose,  2012).  In  sum,  this  research  has  shown  the 
TAPAS  to  be  a  viable  assessment  tool  with  the  potential  to  enhance  new  Soldier  selection.  In 
addition,  preliminary  results  also  suggest  that  the  TAPAS  scales  may  be  useful  for  MOS 
qualification  (Nye,  Drasgow,  Chernyshenko,  et  al.,  2012). 


Table  1.  TAPAS  Dimensions  Assessed  in  the  NSAB 


TAPAS  Facet  Name 

Brief  Description 

“Big  Five” 
Broad 
Factor 

Dominance 

High  scoring  individuals  are  domineering,  “take  charge”  and  are  often  referred  to  by 
their  peers  as  "natural  leaders." 

Extraversion 

Sociability 

High  scoring  individuals  tend  to  seek  out  and  initiate  social  interactions. 

Attention  Seeking 

High  scoring  individuals  tend  to  engage  in  behaviors  that  attract  social  attention;  they 
are  loud,  loquacious,  entertaining,  and  even  boastful. 

Selflessness 

High  scoring  individuals  are  generous  with  their  time  and  resources. 

Agreeableness 

Cooperation 

High  scoring  individuals  are  trusting,  cordial,  non-critical,  and  easy  to  get  along  with. 

Consideration 

Individuals  scoring  high  on  this  facet  are  affectionate,  compassionate,  sensitive,  and 
caring. 

Achievement 

High  scoring  individuals  are  seen  as  hard  working,  ambitious,  confident,  and 
resourceful. 

Conscientiousness 

Order 

High  scoring  individuals  tend  to  organize  tasks  and  activities  and  desire  to  maintain 
neat  and  clean  surroundings. 

Responsibility 

High  scoring  individuals  are  dependable,  reliable  and  make  every  effort  to  keep  their 
promises. 

Self-Control 

High  scoring  individuals  tend  to  be  cautious,  levelheaded,  able  to  delay  gratification, 
and  patient. 

Non-Delinquency 

High  scoring  individuals  tend  to  comply  with  rules,  customs,  norms,  and 
expectations,  and  they  tend  not  to  challenge  authority. 

Adjustment 

High  scoring  individuals  are  worry  free,  and  handle  stress  well;  low  scoring 
individuals  are  generally  high  strung,  self-conscious  and  apprehensive. 

Emotional  Stability 

Even  Tempered 

High  scoring  individuals  tend  to  be  calm  and  stable.  They  don’t  often  exhibit  anger, 
hostility,  or  aggression. 

Optimism 

High  scoring  individuals  have  a  positive  outlook  on  life  and  tend  to  experience  joy 
and  a  sense  of  well-being. 
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Table  1.  (continued) 


Ingenuity 

High  scoring  individuals  are  inventive  and  think  "outside  of  the  box". 

Openness  To 
Experience 

Intellectual  Efficiency 

High  scoring  individuals  are  able  to  process  information  quickly  and  would  be 
described  by  others  as  knowledgeable,  astute,  and  intellectual. 

Tolerance 

High  scoring  individuals  are  interested  in  other  cultures  and  opinions  that  may  differ 
from  their  own.  They  are  willing  to  adapt  to  novel  environments  and  situations. 

Physical  Conditioning 

High  scoring  individuals  tend  to  engage  in  activities  to  maintain  their  physical  fitness 
and  are  more  likely  to  participate  in  vigorous  sports  or  exercise. 

Other 

Purpose  of  the  Current  Research 

TAPAS  scales  have  been  validated  for  use  with  applicants  in  accessioning,  but  have  not 
been  validated  for  use  with  NCO  in-service  testing  and  selection.  The  primary  objective  of  this 
research  was  to  explore  the  validity  of  the  NSAB  for  NCO  in-service  testing  and  selection  into 
the  recruiter  special  duty  assignment.  Research  conducted  on  the  WAI  provided  important 
information  regarding  the  key  dimensions  that  predict  recruiter  perfonnance.  With  this  in  mind, 
TAPAS  scales  were  selected  for  the  NSAB  that  measure  the  key  attributes  from  the  WAI  and 
dimensions  thought  to  be  relevant  to  recruiter  and  NCO  performance. 

This  report  describes  the  approaches  that  were  taken  to  evaluate  the  usefulness  of  the 
NSAB  in  two  separate  samples.  First,  we  examined  the  construct  validity  of  the  NSAB  by 
comparing  it  to  the  WAI  in  a  sample  of  NCOs.  Data  collected  in  this  sample  was  also  used  to 
explore  relationships  between  the  NSAB  and  measures  of  NCO  perfonnance  and  adjustment  to 
Anny  life.  Next,  we  examined  whether  the  NSAB  is  related  to  perfonnance  and  job  attitudes  in  a 
sample  of  experienced  recruiters. 
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CHAPTER  2:  NSAB  CONSTRUCT  VALIDATION 


In  this  chapter  we  describe  the  relationship  between  the  WAI  and  NSAB.  Previous 
research  demonstrated  that  the  WAI  predicted  several  important  criteria,  including  attrition  from 
recruiting  training,  recruiter  performance  and  production  (Horgen  et  ah,  2006;  White  et  ah, 

2001).  As  mentioned  in  the  previous  chapter,  the  NSAB  measures  some  of  the  same  constructs 
as  the  WAI  and  can  be  administered  in  an  online,  unproctored  environment.  Therefore,  our  first 
goal  was  to  examine  the  overlap  between  key  dimensions  on  the  NSAB  and  the  WAI  in  an  NCO 
sample.  Utilizing  this  sample  allowed  us  to  investigate  how  the  NSAB  would  perform  in  an  NCO 
population,  as  recruiters  volunteer  or  are  selected  from  this  population. 

We  begin  by  describing  the  measures,  NCO  sample,  data  collection  procedures,  and 
database  development.  This  is  followed  by  a  description  of  the  psychometric  properties  of  the 
WAI  and  NSAB.  Finally,  we  present  the  results  of  the  relationship  between  the  WAI  and  NSAB 
scales.  Preliminary  results  on  the  relationship  between  the  NSAB  and  NCO  criterion  measures 
are  presented  in  Appendix  A. 

Description  of  NCO  Predictor  and  Criterion  Measures 
NCO  Predictor  Measures 

Two  predictor  measures  were  administered  to  NCOs:  the  WAI  and  the  NSAB. 

Warrior  Attributes  Inventory  (WAI).  The  original  WAI  was  based  on  the 
Noncommissioned  Officer  Leadership  Inventory  (NLSI)  developed  by  ARI  and  USAREC. 
Extensive  research  on  the  NLSI  suggested  that  the  measure  predicted  individual  recruiter 
production  and  attrition  from  recruiter  training  (Horgen  et  ah,  2006).  The  WAI  scores  were  the 
basis  of  a  composite  score  that  was  used  to  construct  an  order  of  merit  list  for  recruiter  screening 
(Halstead,  2009). The  WAI  is  composed  of  two  parts.  Part  I  is  a  125-item  self-report 
questionnaire  that  measures  prior  behaviors  and  reactions  to  specific  life  events  that  are 
indicative  of  such  areas  as  leadership,  interpersonal  skills,  and  integrity.  Part  II  is  a  34-item  self- 
descriptive  inventory  assessing  personality-like  traits  relevant  to  military  performance  including 
work  motivation,  agreeableness,  dependability,  and  dominance.  Each  item  consists  of  four 
behavioral  statements  that  represent  different  personality  constructs.  Within  each  tetrad, 
examinees  are  asked  to  select  one  statement  that  is  most  like  them  and  a  different  statement  that 
is  least  like  them.  Due  to  content  overlap  with  other  operational  tests,  the  WAI  could  not  be 
administered  in  an  unproctored  environment.  To  improve  access  for  NCO  testing,  ARI  and 
USAREC  created  a  plan  to  develop  a  new  measure  (NSAB)  that  could  be  administered  via  the 
web  in  an  unproctored  setting. 

Noncommissioned  Officer  Special  Assignment  Battery  (NSAB).  As  described  in 
Chapter  1,  the  NSAB  is  based  on  the  TAPAS  and  is  a  non-cognitive  assessment  of  18  personality 
dimensions.  This  measure  was  specifically  developed  to  resist  faking  by  administering  pairs  of 
statements  that  represent  different  personality  constructs  and  asking  respondents  to  choose  the 
statement  that  best  describes  him  or  her.  The  key  to  this  approach  was  using  the  MDPP  format 
for  test  construction  and  scoring.  When  fonning  pairs  for  the  MDPP  fonnat,  TAPAS  balances 
the  two  statements  in  terms  of  social  desirability  and  extremity  on  the  dimensions  they  assess.  A 
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difficult  measurement  issue  was  solved  by  adding  a  small  number  of  unidimensional  item  pairs 
in  with  the  multidimensional  pairs  (i.e.,  the  MDPP  items),  which  are  needed  to  identify  the  latent 
trait  metric  and  yield  normative  scores  using  the  MDPP  fonnat  (Stark,  2002;  Stark, 
Chernyshenko,  &  Drasgow,  2005).  TAPAS  scoring  is  then  based  on  the  MDPP  item  response 
theory  model  originally  proposed  by  Stark  (2002).  A  series  of  equations  are  solved  numerically 
to  produce  a  vector  of  latent  trait  scores  for  each  respondent  as  well  as  standard  errors.  The 
static,  non-adaptive  form  of  the  NSAB  was  administered  either  electronically  or  via  paper-and- 
pencil  to  NCOs  because  computer  access  was  not  always  available  and  we  wanted  to  use  the 
same  static  form  across  all  nine  sites. 

NCO  Criterion  Measures 

The  NCO  criterion  measures  are  described  in  Appendix  A. 


NCO  Data  Collection 


NCO  Participants 

Predictor  (WAI,  NSAB)  and  criterion  data  (Anny  Life  Questionnaire  [ALQ], 
performance  rating  scales)  were  collected  from  348  NCOs  and  313  supervisors  from  March 
through  September  2010.  Data  were  obtained  from  NCOs  at  nine  different  sites,  including  Forts 
Hood,  Drum,  Richardson,  Gordon,  Sill,  Bragg,  Carson,  and  Benning,  and  Schofield  Barracks. 
Summary  information  regarding  key  demographic  characteristics  of  the  participants  is  presented 
in  Appendix  A,  Tables  A-l  through  A-4.  The  majority  of  the  participants  were  Caucasian 
(51.7%)  males  (84.5%)  with  an  average  of  approximately  seven  years  in  the  Anny.  ARI 
requested  participation  from  NCOs  (E-5s  and  E-6s)  and  their  first  line  supervisors.  We  expanded 
the  pool  of  participants  to  include  E-4s  to  meet  the  sample  size  requirement  for  the  analyses. 
Thus,  the  majority  of  our  sample  consists  of  E-4s  (19.8%),  E-5s  (59.2%),  and  E-6s  (12.1%). 

Data  Collection  Procedures  -  NCO  Sample 

All  of  the  assessments  were  collected  in  proctored  testing  sessions  and  were  administered 
electronically  or  with  paper-and-pencil  measures,  depending  on  the  facilities  that  were  available. 
PDRI  proctors  used  a  test  administration  protocol  to  ensure  that  standardized  test  administration 
procedures  were  used  across  facilities  and  testing  modalities  and  that  participant  information  and 
predictor  measures  were  kept  secure  at  all  times. 

NCOs  participated  in  the  testing  sessions  for  about  2  hours  and  their  supervisors 
participated  for  about  30  minutes.  At  the  beginning  of  each  predictor  testing  session,  NCOs 
listened  to  a  short  briefing  and  then  completed  a  privacy  act  and  consent  fonn  outlining  the 
details  and  purpose  of  the  research.  Finally,  NCOs  completed  four  measures:  1)  a  demographic 
and  background  infonnation  fonn;  2)  the  NSAB;  3)  the  WAI;  and  4)  the  ALQ. 

Performance  rating  data  were  collected  from  the  first  line  supervisors  identified  by  the 
NCO  participants.  NCOs  were  rated  by  only  one  supervisor.  Supervisors  listened  to  a  short 
briefing  and  then  completed  a  privacy  act  and  consent  fonn  outlining  the  details  and  purpose  of 
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the  research.  Before  supervisors  completed  the  rating  task,  we  conducted  a  rater  training  session 
that  was  designed  to:  1)  orient  raters  to  the  rating  task;  2)  familiarize  raters  with  the  performance 
dimensions  and  how  each  is  defined;  3)  train  raters  to  use  the  behavioral  anchors  to  make  their 
perfonnance  ratings;  4)  describe  common  rater  errors  (e.g.,  halo)  and  how  to  avoid  them;  and  5) 
encourage  raters  to  be  as  accurate  as  possible.  Supervisors  then  completed  their  performance 
ratings  of  subordinates. 

Database  Development 

Separate  databases  were  first  developed  for  the  WAI,  NSAB,  ALQ,  and  supervisor  rating 
data  for  each  post.  Data  from  paper  and  pencil  administrations  were  combined  with  data 
collected  electronically  and  a  final  database  was  created  for  the  key  analyses. 

Database  Cleaning 

We  used  several  steps  to  clean  and  eliminate  poor-quality  data  from  the  database.  The 
measures  collected  using  paper  and  pencil  were  scanned  and  then  combined  with  the  electronic 
measures.  We  checked  the  data  for  impossible  values,  random  responding,  lack  of  variance  in 
responding,  and  missing  data.  Information  from  the  data  collection  problem  logs  was  also  used  to 
identify  poor  quality  data.  A  total  of  16  cases  were  removed  due  to  poor  quality,  missing  data, 
etc.  (see  Table  2).  A  total  of  294  cases  had  data  on  all  of  the  predictor  and  criterion  measures. 


Table  2.  Final  Sample  Sizes 


Measure 

#  Cases  Dropped 

Final  N 

WAI 

14 

324 

NSAB 

0 

342 

ALQ 

0 

336 

Supervisor  Rating 

2 

313 

TOTAL 

16 

294 

NSAB  and  WAI  Results 

Our  primary  goal  in  the  NCO  data  collection  was  to  investigate  the  construct  validity  of 
the  NSAB  by  examining  the  relationship  between  the  WAI  and  NSAB  scales.  Means,  standard 
deviations,  and  correlations  between  the  NSAB  scales  are  shown  in  Table  3.  Table  4  provides  the 
correlations  between  the  NSAB  scales,  WAI  Part  I,  WAI  Part  II,  and  the  WAI  Recruiter 
Selection  Composite. 

Table  4  shows  that  a  number  of  the  NSAB  scales  were  significantly  correlated  with  the 
scales  in  the  WAI  Parts  I  and  II.  For  example,  the  NSAB  Dominance  scale  was  correlated  .50 
with  the  WAI  Part  II  Leadership  Scale,  and  NSAB  Physical  Conditioning  was  correlated  .46  with 
the  WAI  Part  II  Physical  Conditioning  scale.  In  addition,  the  NSAB  Sociability  scale  was 
correlated  .46  with  the  WAI  Part  I  Sociability  scale,  and  NSAB  Dominance  was  correlated  .43 
with  the  WAI  Part  I  Emergent  Leadership  scale.  A  number  of  other  NSAB  facets  also  had 
significant  correlations  with  the  WAI  scales  in  Parts  I  and  II.  Thus,  these  scales  share  similar 
content. 
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Most  importantly,  Table  4  also  shows  that  the  NSAB  scales  were  significantly  correlated 
with  the  WAI  Recruiter  Selection  Composite  which  was  used  to  develop  an  order  of  merit  list  to 
select  NCOs  for  recruiting  duty  (Halstead,  2009).  The  WAI  composite  was  correlated  .39  with 
the  NSAB  Dominance  scale,  .26  with  the  Achievement  scale,  and  .26  with  Intellectual 
Efficiency.  Because  the  current  composite  represents  a  combination  of  several  WAI  scales,  we 
also  examined  the  regression  of  the  WAI  composite  onto  the  NSAB  scales.  The  multiple  R  for 
this  analysis  was  .49  and  the  adjusted  R  was  .44,  indicating  that  a  combination  of  the  NSAB 
scales  was  strongly  correlated  with  the  current  composite. 

The  criterion  measure  results  and  preliminary  validity  analyses  are  presented  in 
Appendix  A. 
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Table  3.  Means,  Standard  Deviations,  and  Correlations  Between  the  NSAB  Scales 


1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

12 

13 

14 

15 

16 

17 

18 

1.  Cooperation 

- 

2.  Dominance 

-.09 

- 

3.  Even-Tempered 

.14 

-.18 

- 

4.  Attention-Seeking 

.12 

.18 

-.11 

- 

5.  Selflessness 

.10 

-.09 

.14 

.03 

- 

6.  Achievement 

-.17 

.29 

-.14 

.07 

-.15 

- 

7.  Ingenuity 

-.10 

.30 

-.13 

.05 

-.06 

.15 

- 

8.  Intellectual  Efficiency 

-.14 

.25 

-.06 

.05 

-.12 

.21 

.22 

- 

9.  Adjustment 

-.03 

.09 

.06 

.04 

-.01 

-.01 

.17 

.13 

- 

10.  Order 

-.10 

.00 

-.13 

-.07 

-.11 

.17 

.02 

-.05 

-.12 

- 

11.  Physical  Conditioning 

-.03 

.20 

-.09 

.08 

-.10 

.31 

.01 

.05 

-.03 

.09 

- 

12.  Responsibility 

-.14 

.20 

.04 

-.18 

.13 

.21 

.05 

.07 

-.01 

.08 

.08 

- 

13.  Self-Control 

-.06 

-.11 

.23 

-.16 

.04 

.03 

-.18 

.08 

-.05 

.04 

.03 

.13 

- 

14.  Sociability 

.06 

.32 

-.13 

.29 

-.03 

.11 

.21 

.15 

.12 

.05 

.02 

.09 

-.25 

- 

15.  Tolerance 

.00 

-.04 

.18 

-.02 

.25 

.05 

.04 

.01 

-.01 

-.08 

.08 

.09 

.04 

.04 

- 

16.  Non-Delinquency 

.20 

-.15 

.11 

-.12 

.14 

.01 

-.23 

-.18 

-.15 

.05 

-.01 

-.06 

.15 

-.10 

.07 

- 

17.  Consideration 

.29 

-.08 

.12 

.19 

.32 

-.14 

-.01 

-.19 

-.12 

.00 

-.06 

.02 

-.14 

.08 

.18 

.08 

- 

18.  Optimism 

.12 

.30 

.07 

.11 

-.05 

.12 

.11 

.15 

.24 

-.11 

.26 

-.01 

.09 

.22 

.01 

-.06 

-.04 

- 

Mean 

-.57 

.23 

.21 

-.24 

-.64 

.23 

.10 

.21 

.27 

-.16 

.39 

-.32 

-.07 

-.14 

-1.01 

-.27 

-.28 

.10 

Standard  Deviation 

.52 

.64 

.53 

.62 

.57 

.61 

.73 

.71 

.66 

.68 

.71 

.57 

.69 

.64 

.71 

.71 

.65 

.78 

Note.  Bold  values  are  significant  at  the  .05  level.  The  sample  size  for  all  of  the  correlations  presented  in  this  table  was  342. 


Table  4.  Correlations  Between  the  NSAB  and  WAI  Part  I  and  Part  II  Scales 


WAI  Scales 


Validity 

Work  Orientation 

Leadership 

Dependability 

Adjustment 

Agreeableness 

Physical  Conditioning 

Lie  Scale 

Emergent  Leadership 

Social  Perceptiveness 

Sociability 

Hostility 

Self-Esteem 

Empathy 

Work  Motivation 

Scores  on  the  Current 

WAI  Composite 

Cooperation 

.03 

-.05 

-.08 

.04 

.00 

.19 

-.10 

.05 

-.12 

.02 

.17 

-.03 

-.15 

.14 

-.13 

-.06 

Dominance 

.11 

.21 

.50 

-.13 

.16 

-.20 

.18 

.18 

.43 

.18 

.23 

.15 

.32 

-.19 

.30 

.39 

Even-Tempered 

.09 

.02 

-.11 

.23 

.21 

.37 

-.01 

-.04 

-.03 

.01 

.06 

-.23 

-.01 

.18 

-.01 

-.08 

Attention-Seeking 

.05 

-.01 

.14 

-.25 

.11 

-.04 

.04 

.00 

.25 

.21 

.41 

.12 

.06 

.05 

.08 

.13 

Selflessness 

-.02 

.10 

.00 

.09 

.06 

.22 

.00 

.05 

-.01 

.07 

.14 

-.09 

-.04 

.41 

.13 

-.04 

Achievement 

.07 

.41 

.30 

.00 

.04 

-.12 

.13 

.10 

.30 

.14 

.10 

.05 

.29 

-.11 

.36 

.26 

Ingenuity 

.02 

.06 

.22 

-.20 

.04 

-.10 

-.06 

-.08 

.30 

.16 

.21 

.20 

.17 

-.11 

.25 

.23 

Intellectual  Efficiency 

.06 

.23 

.28 

-.02 

.15 

-.08 

.05 

.09 

.31 

.12 

.10 

.07 

.34 

-.11 

.27 

.26 

Adjustment 

.05 

-.03 

.02 

-.13 

.32 

-.11 

.02 

.07 

.06 

-.05 

.07 

.04 

.12 

-.21 

.06 

.03 

Order 

.16 

.06 

-.04 

.04 

-.23 

.00 

-.01 

.09 

.02 

.07 

-.06 

.03 

-.02 

.00 

.05 

.01 

Physical  Conditioning 

.09 

.19 

.13 

.10 

.12 

-.02 

.46 

.05 

.16 

.10 

.10 

-.06 

.23 

-.07 

.16 

.16 

Responsibility 

.03 

.22 

.23 

.14 

.06 

.07 

.10 

.08 

.14 

.15 

-.02 

.02 

.13 

.08 

.16 

.09 

Self-Control 

.03 

.11 

-.09 

.22 

.00 

.19 

.01 

-.01 

-.12 

-.01 

-.24 

-.23 

.04 

.07 

-.01 

-.07 

Sociability 

-.06 

.06 

.37 

-.19 

.15 

-.14 

.04 

.07 

.32 

.19 

.46 

.13 

.12 

-.07 

.13 

.23 

Tolerance 

.01 

.01 

.02 

.03 

.05 

.17 

-.01 

.09 

.10 

.17 

.15 

-.06 

.07 

.19 

.11 

.01 

Non-Delinquency 

.12 

.00 

-.18 

.24 

-.07 

.10 

.03 

.10 

-.21 

-.09 

-.14 

-.17 

-.19 

.11 

-.11 

.01 

Consideration 

-.03 

-.02 

-.03 

-.05 

-.01 

.21 

-.07 

.07 

.04 

.20 

.24 

.10 

-.04 

.41 

.02 

-.01 

Optimism 

-.01 

.18 

.24 

.01 

.38 

.06 

.20 

.02 

.25 

.09 

.16 

-.15 

.24 

-.07 

.21 

.12 

Note.  Bold  values  are  significant  at  the  .05  level.  The  sample  sizes  for  these  correlations  range  from  322  to  335. 


Summary 


Our  primary  focus  in  the  NCO  data  collection  was  to  confirm  that  the  NSAB 
demonstrated  significant  overlap  with  the  key  dimensions  on  the  WAI  that  had  predicted 
recruiter  performance  in  previous  research.  The  correlations  between  the  NSAB  and  the  WAI 
scales  indicate  that  the  two  measures  cover  similar  content,  especially  on  the  key  scales  that  were 
related  to  recruiter  performance  in  previous  research.  Further,  when  comparing  the  WAI  to  the 
NSAB  validities,  several  NSAB  scales  outperformed  the  WAI  recruiter  selection  composite  (see 
Appendix  A,  Table  A- 12).  These  results  suggest  that  the  NSAB  is  a  highly  promising  measure 
for  predicting  recruiter  perfonnance.  Our  next  step  was  to  examine  the  validity  of  the  NSAB  in  a 
sample  of  recruiters  to  more  directly  test  the  relationships  between  the  NSAB  scales  and  recruiter 
success. 
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CHAPTER  3:  NSAB  CRITERION  VALIDATION  FOR  RECRUITERS 


In  this  chapter  we  describe  the  methodology  and  results  of  analyses  examining  the 
potential  of  the  NSAB  to  predict  recruiter  perfonnance,  commitment,  job  stress,  and  fit  with  the 
recruiting  job.  We  first  describe  the  methodology  used  for  this  research.  We  then  present  the 
correlation  and  regression  analyses  that  were  used  to  examine  the  validity  of  the  measure. 

We  first  conducted  a  pilot  test  with  students  in  the  Army  Recruiting  Course  at  the 
Recruiting  and  Retention  School  (RRS).  The  Army  Recruiter  Course  (ARC)  is  designed  to  teach 
NCOs  the  skills  needed  for  recruiting  duty.  All  recruiters  must  successfully  complete  the  course 
before  they  begin  recruiting. 

Next,  we  collected  data  from  experienced  recruiters  operating  in  the  field.  Experienced 
recruiters  were  defined  for  the  purposes  of  this  research  as  detailed  recruiters  who  had  been  on 
recruiting  duty  for  between  16  and  35  months.  We  selected  this  range  because  recruiting  experts 
indicated  that,  at  16  months  on  recruiting  duty,  recruiters  are  fully  trained  and  will  have  reached 
their  full  perfonnance  levels.  Also,  previous  research  indicated  that  recruiter  perfonnance 
measures  become  more  reliable  after  about  a  year  on  recruiting  duty  (Horgen  et  ah,  2006).  At  36 
months,  recruiters  either  convert  their  MOS  to  79R  or  return  to  their  previous  MOS  or  another 
duty  assignment. 

Predictor  data  were  collected  from  both  ARC  students  and  experienced  recruiters,  and 
criterion  data  were  collected  on  experienced  recruiters  (see  Table  5).  Predictor  and  criterion  data 
were  collected  via  web-based  instruments  hosted  by  USAREC  and  ARI.  While  our  primary 
purpose  was  to  collect  data  from  experienced  recruiters,  we  also  collected  data  from  ARC 
students  to:  1)  pilot  test  the  online  predictor  measures  to  ensure  that  they  were  working  properly 
before  administering  to  recruiters  in  the  field,  and  2)  provide  data  to  compare  score  distributions 
with  the  experienced  recruiter  sample.  The  ARC  student  pilot  test  is  described  in  detail  in 
Appendix  B. 


Table  5.  Online  Instruments  Administered  to  the  Recruiting  Samples 


ARC  Students 

Detailed  Recruiters 

NSAB  (online  version) 

X 

X 

Recruiting  Life  Questionnaire 

- 

X 

Performance  Rating  Scales 

- 

X 

Description  of  Recruiter  Predictor  and  Criterion  Measures 
Recruiter  Predictor  Measure 


The  computerized  adaptive  version  of  the  NSAB  was  administered  to  both  the 
experienced  and  student  recruiter  samples.  Because  the  NSAB  is  described  in  detail  in  Chapter  2, 
we  do  not  describe  it  again  here. 
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Recruiter  Criterion  Measures 


Two  instruments  were  used  to  collect  criterion  data  from  experienced  recruiters,  the 
Anny  Recruiter  Performance  Rating  Scales  (Appendix  C)  and  the  Recruiting  Life  Questionnaire 
(Appendix  D).  To  develop  the  criterion  measures,  we  conducted  two  workshops  in  June  and 
October  2010.  In  June,  we  were  also  briefed  by  the  Commandant,  Command  Sergeant  Major, 
and  other  senior  personnel  at  the  Recruiting  and  Retention  School  regarding  recruiter  training, 
the  current  recruiting  environment,  and  recruiting  job  requirements.  In  the  June  workshop,  we 
conducted  in-depth  interviews  with  five  ARC  instructors  at  the  Recruiting  and  Retention  School 
at  Fort  Jackson,  South  Carolina.  Instructors  from  the  ARC  were  used  as  subject-matter  experts 
(SMEs)  because  they  are  chosen  from  among  the  top  recruiters  in  the  field  and,  therefore,  have  a 
thorough  understanding  of  the  necessary  performance  requirements  of  the  recruiter  job.  During 
the  interviews,  we  gathered  information  on  the  current  recruiting  environment,  recruiter  tasks 
and  procedures,  and  recruiter  perfonnance  requirements.  ARC  instructors  also  reviewed  a  set  of 
recruiter  rating  scales  developed  in  2001  (Borman  et  al.,  2001).  Thus,  we  developed  behavior- 
based  rating  scales  for  Army  recruiters  by  integrating  current  Army  recruiter  performance 
information  with  previous  Army  recruiter  rating  scale  development  efforts. 

To  begin  the  process  of  updating  the  original  rating  scales,  we  asked  the  instructor  SMEs 
to  determine  the  appropriateness  of  the  original  Army  recruiter  performance  categories  for 
evaluating  the  performance  of  current  Army  recruiters  and  to  identify  where  changes  to  the 
behavioral  definitions  of  the  original  categories  might  be  necessary.  Each  instructor  was  given  a 
copy  of  the  original  Anny  rating  scales  and  asked  to  detennine  whether  the  existing  performance 
categories  were  still  relevant  for  the  current  recruiter  job  and  to  identify  tenninology  differences 
between  the  existing  scales  and  current  practice.  The  instructors  agreed  that  the  existing 
categories  were,  in  general,  relevant  for  current  recruiter  perfonnance.  However,  the  instructors 
did  suggest  some  revisions,  including  terminology  updates,  revisions  due  to  changes  in  recruiting 
technology,  a  change  in  focus  from  “sales”  to  “counseling  and  mentoring”,  and  changes  related 
to  moving  from  the  Delayed  Entry  Program  to  the  Future  Soldier  Training  Program. 

In  October  2010,  we  conducted  additional  workshops  with  six  Recruiting  and  Retention 
School  division  chiefs  and  the  Command  Sergeant  Major  to  provide  an  additional  review  of  the 
Anny  Recruiter  Performance  Rating  Scales.  There  were  only  very  minor  wording  changes 
suggested  during  this  review  and  we  revised  the  scales,  as  necessary,  to  create  the  final  set  of 
rating  scales. 

During  the  June  and  October  2010  workshops  we  also  asked  the  SMEs  to  review  the 
Recruiting  Life  Questionnaire.  This  questionnaire  was  designed  to  collect  recruiter  demographic, 
background,  and  experience  infonnation,  as  well  as  criterion-related  information  regarding 
recruiter  satisfaction,  fit,  and  commitment.  We  developed  new  items  and  adapted  several  items 
from  the  Anny  Life  Questionnaire  for  use  in  a  recruiting-specific  environment.  For  example, 

“My  MOS  is  a  good  match  for  me”  was  changed  to  “Recruiting  is  a  good  match  for  me”. 
Recruiting  instructor  SMEs  reviewed  the  questionnaire  and  provided  clarification  on  recruiting- 
specific  terminology,  and  background,  experience,  awards,  and  training  items.  Based  on  their 
recommendations,  several  modifications  were  made  to  the  questionnaire  content.  The  final 
versions  of  the  two  criterion  instruments  are  described  below. 
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Army  Recruiter  Performance  Rating  Scales.  The  Army  Recruiter  Performance  Rating 
Scales  were  designed  to  be  completed  by  the  supervisors  and  peers  of  recruiters  and  assess 
recruiters’  day-to-day  job  performance.  The  final  Army  Recruiter  Performance  Rating  Scales 
include  the  following  dimensions:  1)  Locating  and  Contacting  Qualified  Prospects;  2)  Gaining 
and  Maintaining  Rapport;  3)  Obtaining  Information  From  and  About  Prospects  and  Making 
Good  Person-Army  Fits;  4)  Counseling/Mentoring  Skills;  5)  Future  Soldier  Training  Program;  6) 
Establishing  and  Maintaining  Good  Relationships  in  the  Community;  7)  Organizing 
Skills/Processing  Skills/Titne  Management;  8)  Supporting  Other  Recruiters  and  USAREC;  and 
9)  Overall  Performance.  Within  each  perfonnance  dimension,  statements  describe  behaviors  to 
anchor  the  10-point  rating  scale.  Raters  were  asked  to  compare  observed  recruiter  behavior  with 
the  statements  on  each  dimension  to  provide  recruiter  job  performance  ratings.  On  each 
dimension,  raters  could  indicate  they  had  not  had  the  opportunity  to  observe  the  recruiter’s 
perfonnance  instead  of  selecting  a  rating. 

Recruiting  Life  Questionnaire.  The  Recruiting  Life  Questionnaire  (RLQ)  is  a  self-report 
measure  that  contains  demographic  and  background  infonnation  items,  as  well  as  recruiter  job 
satisfaction,  recruiting  job  fit  and  affective  commitment,  Anny  affective  commitment, 
satisfaction  with  recruiting  training  and  development  opportunities,  and  recruiting  career 
continuance  items.  Participants  responded  to  the  items  using  a  Likert  scale  (l=strongly  disagree 
to  5=strongly  agree).  In  addition,  there  are  items  regarding  recruiting  awards  received,  reasons 
for  becoming  a  recruiter,  amount  of  time  spent  on  recruiting  duty  overall,  and  amount  of  time 
spent  on  specific  recruiting  tasks. 

Next,  we  describe  the  data  collection  and  results  for  the  experienced  recruiter  sample. 

The  pilot  test  data  collection  procedures  and  results  are  described  in  Appendix  B. 

Experienced  Recruiter  Data  Collection 
Data  Collection  Procedures  -  Experienced  Recruiter  Sample 

Data  were  collected  from  experienced  recruiters  over  a  three  month  period  from  August 
through  October  2011.  All  of  the  data  were  collected  from  experienced  recruiters  and  their  peer 
and  supervisor  raters  through  a  set  of  web-based  assessments.  The  recruiter  assessment  was  split 
into  two  parts:  1)  Part  I,  the  NSAB,  and  2)  Part  II,  consisting  of  background  and  demographic 
questions,  the  Recruiting  Life  Questionnaire,  and  peer  and  supervisor  rater  nominations. 
Recruiters  accessed  Part  I,  the  web-based  NSAB,  through  the  U.S.  Anny  Accessions  Command 
Enterprise  Portal,  and  Part  II  through  ARI’s  survey  administration  site.  Peers  and  supervisors 
accessed  the  web-based  perfonnance  ratings  through  a  separate  ARI  survey  administration  site. 

A  proctored  pilot  test  was  conducted  with  1 1  recruiters  in  late  June  2011  to  verify  that  the 
assessment  instructions  were  clear  and  to  test  the  web-based  data  collection  procedures.  We  also 
conducted  a  pilot  test  of  the  rating  data  collection  in  mid- July  2011  with  the  peers  and 
supervisors  nominated  by  the  recruiters  who  participated  in  the  pilot  test  to  test  the  web-based 
rating  scale  data  collection  procedures. 

The  full  data  collection  began  in  August  2011  with  an  email  announcement  distributed  to 
U.S.  Recruiting  Command  (USAREC)  personnel.  This  was  followed  by  an  email  announcement 
sent  directly  to  2,289  experienced  recruiters  with  between  16  and  35  months  of  recruiting 
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experience.  Email  reminders  were  sent  to  encourage  additional  recruiter  participation.  This  phase 
of  the  data  collection  was  completed  at  the  end  of  October  2011. 

Recruiters  nominated  up  to  three  peers  and  a  station  commander  or  First  Sergeant 
(supervisor  raters)  with  whom  they  had  worked  closely  to  provide  perfonnance  ratings.  These 
raters  received  an  email  in  early  November  identifying  the  recruiter(s)  they  were  being  asked  to 
rate  and  a  link  to  the  web-based  performance  rating  scales.  A  reminder  email  was  sent  to  raters  to 
encourage  additional  participation  and  data  collection  was  completed  at  the  end  of  November 
2011. 

Recruiting  Database  Development  and  Cleaning 

After  the  data  collection  was  completed,  three  databases  were  created  (Part  I,  Part  II,  and 
rating  data).  Next,  we  began  the  process  of  cleaning  and  aggregating  the  data  to  develop  a  final 
database  which  was  used  for  analyses.  Because  recruiters  had  to  access  the  NSAB  (Part  I) 
separately  from  Part  II,  there  were  some  cases  that  had  NSAB  data  only,  and  others  who  had  Part 
II  data  (background,  RLQ,  rater  nominations)  only.  For  those  recruiters  who  did  not  complete 
Part  II,  we  are  missing  demographic,  background,  and  RLQ  data.  We  used  several  steps  to  clean 
and  eliminate  poor-quality  data  from  the  database.  Each  database  was  checked  for  impossible 
values,  random  responding,  lack  of  variance  in  responding,  missing  data,  or  multiple  responses 
(i.e.,  recruiters  or  raters  who  completed  the  instruments  more  than  once).  We  also  screened  out 
individuals  who  endorsed  two  items  that  we  created  to  check  for  careless  responding  (recruiters 
who  reported  they  had  recruited  less  than  two  people  in  the  last  year  or  had  not  attended  any 
NCOES  schools).  All  of  the  recruiters  in  our  sample  would  have  attended  an  NCOES  school  and 
would  have  recruited  two  or  more  individuals  into  the  Anny  within  the  year. 

Detailed  Recruiter  Demographics  and  Background 

Demographic  and  background  data  on  the  experienced  recruiter  participants  are  presented 
in  Appendix  E,  Tables  E-lthrough  E-3.  Demographic  data  is  available  for  only  those  recruiters 
who  completed  Part  II  (N=854).  The  majority  of  the  participants  were  white  (74.7%),  male 
(94.0%),  and  had  some  college  education  (60.9%).  Most  participants  were  E-5s  (25.0%)  or  E-6s 
(58.8%)  with  an  average  of  about  eleven  years  in  the  Army  and  just  over  two  years  in  recruiting. 
All  six  recruiting  brigades  were  represented  in  our  sample.  Over  75%  of  the  participants  were 
selected  (vs.  volunteered)  for  recruiting  duty.  Finally,  57.9%  of  the  experienced  recruiters 
operated  under  the  team  recruiting  system,  33.2%  operated  under  the  Legacy  recruiting  system, 
and  8.9%  operated  under  the  Pinnacle  recruiting  system  (see  Table  6).  In  the  Legacy  recruiting 
system,  each  recruiter  is  responsible  for  every  recruiting  task  and  has  to  meet  individual 
recruiting  objectives.  In  the  team  and  Pinnacle  recruiting  systems,  recruiters  are  aligned  to 
recruiting  roles  (e.g.,  prospecting,  processing)  that  focus  on  a  smaller  range  of  tasks  allowing 
each  recruiter  to  specialize  in  one  area  of  recruiting.  The  Pinnacle  system  is  a  team-based 
structure  with  team  recruiting  goals.  We  also  had  representation  from  small,  medium,  and  large 
size  recruiting  stations. 
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Table  6.  Recruiter  Entry  Status,  Company  Recruiting  System, 
and  Station  Size 


N 

% 

Recruiter  Entry  Status 

Volunteered  Thru  Recruit-the-Recruiter  Team 

141 

16.8 

Volunteered  Thru  Branch 

64 

7.6 

Selected  for  Recruiting  Duty 

634 

75.6 

Missing 

15 

... 

Total 

854 

100 

Company  Recruiting  System 

Team  Recruiting 

476 

57.9 

Pinnacle 

73 

8.9 

Legacy 

273 

33.2 

Missing 

32 

... 

Total 

854 

100 

Number  of  Recruiters  in  Station 

1-2 

96 

11.6 

3-4 

204 

24.7 

5-6 

263 

31.9 

7  or  more 

262 

31.8 

Missing 

29 

... 

Total 

854 

100 

Recruiters  indicated  that  they  spent  their  time  distributed  across  recruiting  tasks,  with  an 
average  of  40%  time  spent  in  prospecting  tasks  and  somewhat  less  time  in  counseling/mentoring 
(15%),  processing  (20%),  and  Future  Soldier  Training  Program  (16%)  tasks  (see  Table  7).  Most 
recruiters  reported  working  8-9  hours  per  day  (not  including  PT)  (see  Table  8). 


Table  7.  Percent  Time  Spent  in  Recruiting  Tasks 


M 

SD 

Prospecting 

40.09 

23.55 

Counseling/Mentoring 

14.65 

9.98 

Processing 

20.02 

17.48 

Future  Soldier  Training  Program 

16.45 

20.92 

Station  Commander  Duties 

5.49 

11.91 

Other 

3.31 

9.46 

Table  8.  Number  of  Hours  Spent  Recruiting  (per  day*) 

N 

% 

5  or  fewer 

18 

2.2 

6-7 

130 

16.0 

8-9 

433 

53.3 

10-11 

209 

25.7 

12  or  more 

23 

2.8 

Missing 

41 

... 

Total 

854 

100 

‘Note:  total  hours  do  not  include  PT. 
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We  also  asked  respondents  to  describe  the  reasons  they  became  recruiters  by  checking  all 
the  responses  that  applied.  A  large  majority  of  the  participants  reported  that  they  did  not 
volunteer  for  recruiting  duty  (70.2%).  Recruiters  also  responded  that  they  believed  in  the  Army 
and  wanted  to  share  it  with  others  (12.2%),  they  wanted  to  help  young  people  (1 1.8%),  and  they 
thought  recruiting  duty  would  enhance  their  Anny  career  (10.9%)  (see  Table  9). 

Table  9.  If  Volunteered  for  Recruiting  Duty,  Reasons  for  Becoming 


a  Recruiter 


A/a 

% 

1  did  not  volunteer 

543 

70.2 

Able  to  choose  the  location  of  my  duty  station 

70 

9.1 

Wanted  a  change  from  my  MOS 

58 

7.5 

Recruiting  duty  is  career  enhancing 

84 

10.9 

Recruiting  duty  is  necessary  for  promotion 

46 

6.0 

Believe  in  my  Service  and  want  to  share  with  others 

94 

12.2 

Want  to  help  young  people 

91 

11.8 

Ready  for  a  challenge 

76 

9.8 

1  had  no  choice 

54 

7.0 

Wanted  a  change  from  deployment 

63 

8.2 

Participants  could  list  more  than  one  response  (i.e.,  reason) 


Experienced  Recruiter  Criterion  Measure  Results 
Recruiter  Rating  Scales  -  Rater  Demographics 

Recruiters  nominated  peers  and  supervisors  to  provide  performance  ratings  on  nine 
dimensions  of  performance.  A  total  of  887  peer  and  supervisor  raters  provided  ratings  on  637 
recruiters  for  a  total  of  1065  rater-ratee  pairs.  Peer  and  supervisor  rater  demographics  are 
presented  in  Appendix  F,  Tables  F-l  through  F-5.  The  majority  of  the  raters  were  white  (79.4%), 
male  (94.1%),  and  had  some  college  education  (54.9%).  Most  raters  were  E-6s  (39.4%)  or  E-7s 
(40.4%)  and  were  recruiters  (63.3%)  or  station  commanders  (24.2%),  and  had  over  thirteen  years 
of  experience  in  the  Army. 

Recruiter  Performance  Rating  Scales  -  Descriptives 

Most  raters  indicated  they  were  very  familiar  with  the  recruiters’  job  performance 
(86.2%)  (see  Table  10).  There  were  137  rater-ratee  pairs  where  the  rater  indicated  that  he/she 
was  “Not  Very  Familiar”  with  the  recruiter’s  perfonnance.  In  these  cases,  the  raters  were  not 
asked  to  make  performance  ratings  on  these  recruiters.  Table  1 1  illustrates  the  distribution  of 
ratings  across  the  10-point  rating  scale  for  the  nine  perfonnance  dimensions  combined.  There  is 
a  slightly  negative  skew,  as  is  typical  for  perfonnance  ratings,  with  a  lower  percentage  of  ratings 
at  the  less  effective  end  of  the  scale.  However,  while  most  of  the  ratings  fall  in  the  6-10  range, 
there  is  reasonable  variability  in  the  ratings  suggesting  that  raters  were  differentiating  between 
less  and  more  effective  perfonnance  across  dimensions. 
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Table  10.  Rater  Familiarity  with  Recruiter 


N 

% 

Very  Familiar 

918 

86.2 

Somewhat  Familiar 

147 

13.8 

Missing 

0 

— 

Total 

1065 

100 

Table  11.  Number  and  Percentage  of  Ratings  at  Each  Scale  Point 


Rating  Scale  Point 
(l=Lowest  10=Highest) 

Number  of  Ratings 

Percentage  of  Ratings 

1 

106 

1.11 

2 

115 

1.20 

3 

222 

2.32 

4 

319 

3.33 

5 

695 

7.25 

6 

1092 

11.39 

7 

1427 

14.89 

8 

1890 

19.72 

9 

1803 

18.81 

10 

1723 

17.98 

Have  not  had  opportunity  to  observe 

116 

1.21 

Missing  Data 

77 

0.80 

Total  number  of  ratings  across  all  nine  dimensions  (N  =  9,585). 


Means,  standard  deviations,  and  interrater  reliability  estimates  for  each  dimension  of  the 
rating  scales  are  provided  in  Table  12.  There  were  an  average  of  1.67  raters  per  recruiter,  and  the 
interrater  reliability  estimates  indicate  fairly  good  agreement  among  raters,  particularly  given  the 
number  of  raters  per  recruiter. 


Table  12.  Recruiter  Performance  Rating  Scales  -  Means,  SDs  and  ICCs 


Standard 

Dimension 

Mean 

Deviation 

ICCsa 

Locating  and  Contacting  Qualified  Applicants 

7.21 

1.94 

.71 

Gaining  and  Maintaining  Rapport 

7.82 

1.77 

.68 

Obtaining  Information  From  and  About  Prospects  and 

Making  Good  Person-Army  Fits 

7.54 

1.76 

.64 

Counseling/Mentoring  Skills 

7.41 

1.87 

.70 

Future  Soldier  Training  Program 

8.01 

1.58 

.62 

Establishing  and  Maintaining  Good  Relationships  in  the 
Community 

7.60 

1.80 

.63 

Organizing  Skills/Processing  Skills/Time  Management 

7.43 

1.88 

.67 

Supporting  Other  Recruiters  and  USAREC 

7.68 

1.78 

.57 

Overall  Effectiveness/Performance 

7.64 

1.80 

.72 

Samples  sizes  for  the  performance  dimensions  ranged  from  628-639;  aICC(1,k);  mean  number  of  raters  per 
ratee=1.67 
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Recruiter  Performance  Rating  Scales  -  Factor  Analyses 

Correlations  between  the  performance  ratings  are  presented  in  Table  13.  As  shown,  the 
scales  were  highly  correlated.  Therefore,  we  conducted  factor  analyses  to  determine  whether 
these  scales  could  be  reasonably  combined  to  create  a  reduced  number  of  criteria  for  examining 
NSAB  validity. 

Table  13.  Correlations  Between  the  Performance  Rating  Scales  in  the  Recruiter  Sample 


Criteria 

1 

2 

3 

4 

5 

6 

7  8  9 

1.  Locating  and  Contacting  Qualified  Applicants 

- 

2.  Gaining  and  Maintaining  Rapport 

.79 

- 

3.  Obtaining  Information  From  and  About 
Prospects  and  Makinq  Good  P-A  Fits 

.80 

.80 

- 

4.  Counseling/Mentor  Skills 

.84 

.78 

.82 

5.  Future  Soldier  Training  Program 

.63 

.67 

.68 

.68 

- 

6.  Establishing  and  Maintaining  Good 
Relationships  in  the  Community 

.77 

.72 

.76 

.75 

.68 

* 

7.  Organizing,  Processing,  and  Time 
Management  Skills 

.67 

.63 

.72 

.72 

.65 

.65 

- 

8.  Supporting  Other  Recruiters  and  USAREC 

.71 

.66 

.70 

.72 

.65 

.75 

.70 

9.  Overall  Effectiveness/Performance 

.82 

.77 

.79 

.83 

.69 

.76 

.75  .79 

Note.  Bold  values  are  significant  at  the  .05  level.  The  sample  sizes  for  these  correlations  range  from  593  to  606. 
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First,  we  conducted  an  exploratory  factor  analysis  (EFA)  of  the  performance  rating  data. 
The  scree  plot  shown  in  Figure  1  indicates  a  very  strong  first  factor,  indicating  an  essentially 
unidimensional  factor  structure.  In  addition,  a  confirmatory  factor  analysis  (CFA)  also  indicated 
that  a  single  factor  model  fit  the  data  well  (Root  Mean  Square  Error  of  Approximation  =  .11; 
Comparative  Fit  Index  =  .98;  Standardized  Root  Mean  Square  Residual  =  .02).  The  completely 
standardized  factor  loadings  from  this  CFA  model  are  shown  in  Table  14.  Based  on  these  results, 
we  summed  the  nine  performance  ratings  to  form  a  single  variable  and  examined  the  validity  of 
the  NSAB  scales  for  predicting  this  perfonnance  rating  composite. 


Figure  1.  Scree  Plot  of  the  Performance  Rating  Scales  in  the  Sample  of  Experienced  Recruiters. 

Table  14.  Factor  Loadings  from  the  Single  Factor  CFA  Model  of  the  Performance  Rating  Scales  in 
the  Sample  of  Experienced  Recruiters _ 

Completely 
Standardized  Factor 

Performance  Rating  Scales  Loadings 


Locating  and  Contacting  Qualified  Applicants  .90 

Gaining  and  Maintaining  Rapport  .85 

Obtaining  Information  From  and  About  Prospects  .89 

Counseling/Mentor  Skills  .90 

Future  Soldier  Training  Program  .77 

Establishing  and  Maintaining  Good  Relationships  in  the  Community  .85 

Organizing,  Processing,  and  Time  Management  Skills  .81 

Supporting  Other  Recruiters  and  USAREC  .82 

Overall  Effectiveness/Performance  .91 
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Recruiting  Life  Questionnaire  -  Descriptives 


The  RLQ  was  made  up  of  two  sections,  a  multi-choice  item  section  which  includes  items 
on  self-rated  recruiter  perfonnance,  awards  received,  and  recruiter  career  intention.  Results  for 
these  items  are  presented  in  Tables  15  through  17.  The  distribution  of  awards  received  in  the 
sample  was  somewhat  limited  by  the  items  included  on  the  awards  question.  Therefore,  we  did 
not  include  this  criterion  in  the  validity  analyses. 


Table  15.  USAREC  Awards 


/\/a 

% 

NCO  of  the  Year 

15 

1.8 

NCO  of  the  Quarter 

33 

3.9 

Recruiter  of  the  Year 

26 

3.0 

Recruiter  of  the  Quarter 

55 

6.4 

Master  Recruiter  Badge 

49 

5.7 

SGT  Audie  Murphy 

5 

0.6 

Participants  could  check  more  than  one  response 

Table  16.  MOS  Reclassification 
(i.e.,  1  plan  to  reclassify  to  79R) 

N 

% 

Yes 

119 

14.6 

No 

571 

70.2 

Undecided 

116 

14.3 

Already  Reclassified 

7 

.9 

Missing 

41 

... 

Total 

854 

100 

Table  17.  Freedom  to  select  own  assignment 
(i.e.,  If  you  had  the  freedom,  what  would  you  do?) 

N 

% 

Remain  in  recruiting 

231 

28.4 

Return  to  previous  MOS 

358 

44.0 

Select  a  new  MOS 

156 

19.2 

Leave  the  service 

68 

8.4 

Missing 

41 

... 

Total 

854 

100 
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The  other  section  of  the  RLQ  consisted  of  Likert  items  measuring  recruiter  job 
satisfaction,  recruiting  fit  and  affective  commitment,  Army  affective  commitment,  and 
satisfaction  with  recruiting  training  and  development  opportunities.  We  used  a  rational  approach 
to  group  the  RLQ  items  into  four  scales  based  on  the  item  content:  Army  commitment,  recruiter 
job  fit,  recruiting  stress,  and  training  and  development  satisfaction.  Scale  reliabilities  are 
presented  in  Table  18. 


Table  18.  RLQ  Scale  Reliabilities 


Scale 

N  items 

a 

RLQ  Army  Commitment 

2 

.85 

RLQ  Recruiter  Fit 

9 

.94 

RLQ  Recruiting  Stress 

5 

.81 

RLQ  Training  and  Development  Satisfaction 

6 

.82 

Final  Criterion  Measures  -  Experienced  Recruiter  Sample 

Table  19  shows  the  descriptive  statistics  for  the  reduced  number  of  criteria  used  in  the 
validity  analyses.  In  addition,  Table  20  provides  the  correlations  between  these  criteria.  As 
shown  here,  the  perfonnance  ratings  composite  was  significantly  correlated  with  the  RLQ 
composites.  However,  these  correlations  were  generally  small.  Given  the  small  correlations 
between  these  criteria,  different  NSAB  scales  may  predict  each  criterion. 


Table  19.  Descriptive  Statistics  for  the  Criteria  in  the  Recruiter  Sample 


N 

Mean 

Standard 

Deviation 

Min. 

Max 

Performance  Ratings  Composite 

609 

67.46 

14.74 

14.00 

90.00 

RLQ  Army  Commitment 

808 

3.33 

1.14 

1.00 

5.00 

RLQ  Recruiter  Fit 

813 

3.24 

1.02 

1.00 

5.00 

RLQ  Recruiting  Stress 

811 

3.50 

.85 

1.00 

5.00 

RLQ  Training  and  Development 
Satisfaction 

816 

3.42 

.83 

1.00 

5.00 

Table  20.  Correlations  Between  the  Criteria  in  the  Recruiter  Sample 


Criteria 

1  2  3  4  5 

1.  Performance  Ratings  Composite 

- 

2.  RLQ  Army  Commitment 

.14 

3.  RLQ  Recruiter  Fit 

.24  .64 

4.  RLQ  Recruiting  Stress 

-.14  -.41  -.52 

5.  RLQ  Training  and  Development  Satisfaction 

.21  .62  .72  -.49 

Note.  Bold  values  are  significant  at  the  .05  level.  The  sample  sizes  for  these  correlations  range  from  582  to  813. 
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NSAB  Validity  Results:  Experienced  Recruiter  Sample 

Table  21  provides  the  descriptive  statistics  for  the  NSAB  scales,  and  Table  22  provides 
the  correlations  between  these  scales  in  the  sample  of  experienced  recruiters.  Table  23  shows  the 
correlations  between  the  NSAB  scales  and  each  of  the  criteria  in  this  sample  as  well.  As  shown 
in  Table  23,  a  number  of  the  NSAB  scales  were  strong  predictors  of  the  criteria  in  the  sample  of 
experienced  recruiters.  Achievement,  Optimism,  and  Sociability  were  three  of  the  strongest 
predictors  for  many  of  the  criteria.  Optimism  had  the  strongest  correlations  across  all  of  the 
criteria.  However,  Sociability  had  the  largest  correlation  with  a  .40  correlation  with  Recruiting 


Fit. 

Table  21.  Descriptive  Statistics  for  the  NSAB  Scales  in  the  Recruiter  Sample 


N 

Mean 

Standard 

Deviation 

Min. 

Max 

Cooperation 

1032 

-.14 

.47 

-1.57 

1.44 

Dominance 

1032 

.28 

.57 

-1.70 

2.09 

Even-Tempered 

1032 

-.03 

.57 

-1.91 

1.97 

Attention-Seeking 

1032 

-.40 

.54 

-1.79 

1.95 

Selflessness 

1032 

-.19 

.46 

-1.96 

1.66 

Achievement 

1032 

.27 

.62 

-1.57 

2.06 

Ingenuity 

1032 

.13 

.49 

-1.61 

1.88 

Intellectual  Efficiency 

1032 

.26 

.56 

-1.77 

2.02 

Adjustment 

1032 

.09 

.62 

-1.73 

1.98 

Order 

1032 

-.22 

.53 

-1.75 

1.63 

Physical  Conditioning 

1032 

.16 

.62 

-1.78 

2.06 

Responsibility 

1032 

.36 

.46 

-1.06 

1.87 

Self-Control 

1032 

.07 

.53 

-1.73 

1.53 

Sociability 

1032 

-.15 

.66 

-1.90 

1.88 

Tolerance 

1032 

-.13 

.55 

-2.01 

1.87 

Non-Delinquency 

1032 

-.02 

.53 

-2.09 

1.58 

Consideration 

1032 

-.13 

.50 

-1.79 

1.65 

Optimism 

1032 

.22 

.59 

-1.87 

2.28 
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Table  22.  Correlations  Between  the  NSAB  Scales  in  the  Sample  of  Experienced  Recruiters 


NSAB  Scales 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

12 

13 

14 

15 

16 

17  18 

1.  Cooperation 

- 

2.  Dominance 

-.09 

- 

3.  Even-Tempered 

.38 

-.08 

- 

4.  Attention-Seeking 

.04 

.16 

-.03 

- 

5.  Selflessness 

.19 

.05 

.16 

.03 

- 

6.  Achievement 

.04 

.42 

.01 

.12 

.15 

- 

7.  Ingenuity 

.00 

.20 

-.03 

.11 

.11 

.21 

- 

8.  Intellectual  Efficiency 

-.05 

.27 

-.03 

.14 

.03 

.19 

.33 

- 

9.  Adjustment 

-.01 

.17 

.20 

.10 

-.02 

.06 

.13 

.21 

- 

10.  Order 

-.04 

.14 

-.08 

-.01 

-.04 

.21 

.00 

-.02 

-.06 

- 

11.  Physical  Conditioning 

-.04 

.15 

-.04 

.07 

-.02 

.23 

.04 

.01 

.11 

.10 

- 

12.  Responsibility 

.11 

.22 

.04 

.03 

.16 

.33 

.09 

.15 

.06 

.09 

.05 

- 

13.  Self-Control 

.08 

.08 

.21 

-.05 

.04 

.22 

.05 

.14 

.08 

.20 

.03 

.17 

- 

14.  Sociability 

.25 

.23 

.16 

.39 

.20 

.24 

.21 

.11 

.11 

-.03 

.10 

.04 

-.04 

- 

15.  Tolerance 

.24 

.02 

.17 

.09 

.31 

.08 

.14 

.00 

.03 

-.04 

.01 

.10 

.10 

.29 

- 

16.  Non-Delinquency 

.09 

.06 

.05 

-.12 

.09 

.11 

-.10 

-.09 

-.08 

.15 

.01 

.11 

.15 

-.01 

.00 

- 

17.  Consideration 

.32 

.09 

.22 

.18 

.24 

.14 

.06 

.07 

-.01 

.01 

-.01 

.10 

.09 

.35 

.24 

.09 

- 

18.  Optimism 

.14 

.19 

.30 

.14 

.12 

.19 

.10 

.14 

.37 

.04 

.15 

.07 

.13 

.31 

.18 

.06 

.13 

Note.  Bold  values  are  significant  at  the  .05  level.  The  sample  size  for  these  correlations  was  1032. 


Table  23.  Correlations  Between  the  NSAB  Scales  and  the  Criteria  in  the  Sample  of 


Experienced  Recruiters 


Performance 

Rating 

Composite 

RLQ  Army 
Commitment 

RLQ 

Recruiting 

Fit 

RLQ 

Recruiting 

Stress 

RLQ  Training 
and 

Development 

Satisfaction 

Achievement 

.27 

.16 

.25 

-.09 

.14 

Adjustment 

.09 

.04 

.08 

-.18 

.04 

Attention  Seeking 

.12 

.07 

.12 

-.01 

.05 

Consideration 

.11 

.12 

.24 

-.12 

.14 

Cooperation 

.07 

.17 

.19 

-.08 

.18 

Dominance 

.21 

.02 

.13 

-.06 

.02 

Even-Tempered 

.09 

.20 

.23 

-.13 

.22 

Ingenuity 

.03 

-.04 

.03 

-.05 

-.04 

Intellectual  Efficiency 

.12 

-.07 

-.04 

.02 

-.05 

Non-Delinquency 

.10 

.12 

.12 

-.02 

.09 

Optimism 

.29 

.25 

.26 

-.25 

.24 

Order 

.06 

.03 

.05 

.03 

-.02 

Physical  Conditioning 

.09 

.02 

.01 

-.03 

.04 

Responsibility 

.12 

.04 

.07 

-.01 

.04 

Self-Control 

.07 

.06 

.11 

-.05 

.10 

Selflessness 

.03 

.17 

.23 

-.13 

.21 

Sociability 

.14 

.27 

.40 

-.18 

.25 

Tolerance 

-.02 

.21 

.27 

-.18 

.24 

Note.  Bold  values  are  significant  at  the  .05  level.  The  sample  sizes  for  these  correlations  range  from  504  to  670. 

As  noted  above,  individual  recruiters’  performance  was  rated  by  both  their  peers  and 
supervisors.  Correlations  between  the  peer  and  supervisor  ratings  of  the  same  recruiter  on  each 
of  the  nine  rating  dimensions  ranged  from  .33  to  .57  (N=128-135).  However,  it  is  possible  that 
the  NSAB  scales  would  differentially  predict  the  ratings  from  these  different  sources.  Therefore, 
we  also  examined  the  validities  separately  for  peer  and  supervisor  ratings  and  these  validities  are 
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illustrated  in  Table  24.  For  comparison,  correlations  with  the  combined  performance  criterion 
(i.e.,  those  provided  in  Table  23  above)  were  also  included.  As  shown  in  Table  24,  there  were 
some  moderate  differences  in  the  validities.  For  example,  the  correlation  between  Optimism  and 
peer  performance  ratings  was  .24  compared  to  .33  with  supervisor  ratings.  Similarly,  the 
correlation  between  Tolerance  and  peer  ratings  was  -.06  compared  to  .12  with  the  supervisor 
ratings.  Although  the  correlations  with  Tolerance  were  not  significant,  the  variation  in  these 
validities  illustrates  the  potential  differences  between  peer  and  supervisor  ratings.  Because  of  the 
sample  size  limitations  when  examining  peer  and  supervisor  ratings  separately,  we  continue  to 
focus  on  the  combined  performance  ratings  composite  in  subsequent  analyses.  However,  we  also 
explore  differences  across  these  rating  sources  when  applicable. 


Table  24.  Comparing  the  Correlations  Between  the  NSAB  Scales 
and  Peer  or  Supervisor  Performance  Ratings 


NSAB  Scales 

Performance 

Rating 

Composite3 

Peer 

Performance 

Ratings'1 

Supervisor 

Performance 

Ratings0 

Achievement 

.27 

.26 

.34 

Adjustment 

.09 

.10 

.06 

Attention  Seeking 

.12 

.09 

.07 

Consideration 

.11 

.09 

.16 

Cooperation 

.07 

.11 

.04 

Dominance 

.21 

.17 

.25 

Even-Tempered 

.09 

.11 

.15 

Ingenuity 

.03 

-.07 

.07 

Intellectual  Efficiency 

.12 

.04 

.22 

Non-Delinquency 

.10 

.07 

.09 

Optimism 

.29 

.24 

.33 

Order 

.06 

.09 

.04 

Physical  Conditioning 

.09 

.07 

.09 

Responsibility 

.12 

.14 

.07 

Self-Control 

.07 

.04 

.08 

Selflessness 

.03 

-.04 

.02 

Sociability 

.14 

.12 

.15 

Tolerance 

-.02 

-.06 

.12 

Note.  Bold  values  are  significant  at  the  .05  level. 


a  N  =  504. 
bN  =215. 
c  N  =  109. 
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We  also  estimated  the  relative  validity  of  the  NSAB  scales  using  regression  analysis. 
Table  25  shows  the  standardized  weights  for  the  NSAB  scales  that  were  significant  predictors  of 
each  criterion.  Again,  Optimism  and  Tolerance  were  the  most  consistent  predictors  across 
criteria.  Achievement  also  predicted  several  of  the  criteria.  In  addition,  Table  26  reports  the 
differences  in  the  standardized  regression  weights  for  the  NSAB  scales  when  predicting  peer  and 
supervisor  ratings.  Again,  the  results  indicate  moderate  differences  across  these  sources. 


Table  25.  Standardized  Regression  Weights  for  the  NSAB  Scales  that  are 
Significant  Predictors  of  Each  Criterion  in  the  Sample  of  Experienced  Recruiters 


RLQ  Training 

Performance  RLQ  and 

Rating  RLQ  Army  RLQ  Recruiting  Development 

Composite  Commitment  Recruiting  Fit  Stress  Satisfaction 

Achievement 

.20  .12  .15  .10 

Adjustment 

-.14 

Attention  Seeking 

Consideration 

Cooperation 

Dominance 

Even-Tempered 

.08  .09 

Ingenuity 

-.09  -.10 

Intellectual 

Efficiency 

-.09  .08 

Non-Delinquency 

Optimism 

.24  .14  .10  -.16  .14 

Order 

Physical 

Conditioning 

Responsibility 

Self-Control 

Selflessness 

.07  .10 

Sociability 

.19  .27  .16 

Tolerance 

-.09  .10  .12  -.10  .12 

Multiple  R 

.40  .41  .51  .34  .41 

Adjusted  R 

.36  .38  .49  .30  .38 
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Table  26.  Comparing  the  Standardized  Regression  Weights  for  the 
NSAB  Scales  that  are  Significant  Predictors  of  Peer  and  Supervisor 
Performance  Ratings 


NSAB  Scales 

Performance  Peer  Supervisor 

Ratings  Performance  Performance 

Composite3  Ratings15  Ratings0 

Achievement 

.20  .23  .28 

Adjustment 

Attention  Seeking 

Consideration 

Cooperation 

Dominance 

Even-Tempered 

Ingenuity 

Intellectual  Efficiency 

Non-Delinquency 

Optimism 

.24  .18  .33 

Order 

Physical  Conditioning 

Responsibility 

Self-Control 

Selflessness 

Sociability 

Tolerance 

-.09 

Multiple  R 

.40  .40  .49 

Adjusted  R 

.36  .28  .30 

a  N  =  504. 
b  N  =  215. 
c  N  =  109. 


Given  these  results,  the  NSAB  appears  to  be  useful  for  predicting  the  performance  of 
Army  recruiters.  Therefore,  we  next  examined  the  extent  to  which  the  NSAB  scales  could  be 
used  to  select  Soldiers  for  recruiting  duty.  To  do  so,  we  first  combined  the  criteria  into  a  single 
variable  by  creating  a  criterion  composite  using  unit  weights  for  RLQ  Army  Commitment,  RLQ 
Recruiting  Fit,  and  RLQ  Recruiting  Stress,  and  double  weighting  performance  ratings.  The  goal 
of  this  step  was  to  create  a  single  criterion  that  could  be  used  to  develop  a  selection  composite  of 
NSAB  scales.  We  then  regressed  this  criterion  composite  onto  the  NSAB  scales  and  estimated 
the  regression  weights  for  each  scale.  Based  on  these  analyses,  we  identified  the  NSAB  scales 
that  were  significant  predictors  of  the  criterion  and  used  these  scales  to  form  a  composite  for 
recruiter  selection.  The  multiple  R  for  this  model  was  .48  and  the  adjusted  multiple  R  was  .47, 
suggesting  that  the  NSAB  scales  were  strong  predictors  of  the  criterion.  Due  to  sensitivity 
concerns,  detailed  results  are  reported  separately  in  a  limited  distribution  table.  Those  interested 
in  obtaining  a  copy  of  this  table  should  contact  ARI  authors  for  further  information. 
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Using  the  NSAB  scales  and  the  weights,  we  calculated  an  NSAB  composite  score  for 
each  individual  in  the  sample.  Table  27  shows  the  significant  zero-order  correlations  between 
these  predicted  scores  and  the  various  criteria  measured  in  this  dataset.  Overall,  the  NSAB 
composite  was  a  relatively  strong  predictor  of  the  criteria  for  experienced  recruiters.  The  highest 
correlation  for  this  composite  (.44)  was  with  recruiting  fit,  suggesting  that  high  scores  on  the 
NSAB  composite  are  associated  with  better  fit  between  the  Soldier  and  his  or  her  job  as  a 
recruiter.  Individuals  who  receive  high  scores  on  the  NSAB  composite  were  also  likely  to  be 
more  committed  to  the  Army,  satisfied  with  the  training  and  development  opportunities  that  they 
receive,  and  receive  higher  performance  ratings  from  their  peers  and  supervisors.  In  other  words, 
the  composite  developed  here  was  associated  with  a  number  of  important  criteria  for  recruiter 
performance. 

Table  27.  Significant  Correlations  Between  the 
Criterion  Measures  and  the  Predicted  Scores  on  the 
NSAB  Composite 


NSAB 

Criteria  Composite 


Criterion  Composite 

.48 

Performance  Ratings  Composite 

.31 

RLQ  Army  Commitment 

.35 

RLQ  Recruiting  Fit 

.44 

RLQ  Recruiting  Stress 

-.26 

RLQ  Training  and  Development  Satisfaction 

.34 

Note.  The  sample  sizes  for  these  correlations  range  from  504  to  670. 

Figure  2  illustrates  the  practical  importance  of  the  relationships  shown  in  Table  27.  This 
figure  shows  quintile  plots  predicting  Army  commitment,  recruiting  fit,  recruiting  stress,  and  the 
performance  rating  composite  using  the  NSAB  composite  developed  here.  On  the  X-axes  of 
these  plots  are  the  quintiles  of  recruiters  based  on  the  NSAB  composite  score  (i.e.,  the  20%  of 
recruiters  with  the  lowest  NSAB  composite  scores  are  in  the  first  quintile,  etc.).  On  the  Y-axes 
are  average  scores  on  the  criterion  variables.  The  Y-axes  for  these  plots  are  scaled  to  range  from 
+/-  1  standard  deviation  from  the  mean  of  the  criterion. 

As  shown  in  Figure  2,  the  NSAB  composite  was  useful  for  identifying  high  performing 
recruiters.  For  example,  test-takers  in  the  bottom  20%  on  the  NSAB  composite  were  less 
committed  to  the  Army,  experienced  more  recruiting  stress,  were  rated  as  lower  perfonners  by 
their  peers  and  supervisors,  and  had  lower  perceptions  of  fit  with  their  job  than  those  in  the 
highest  20%.  Thus,  the  validities  reported  above  can  have  important  practical  implications  for 
selecting  recruiters. 

Figure  3  illustrates  the  percentages  of  ARC  students  with  scores  on  the  NSAB  composite 
that  correspond  to  the  quintiles  examined  above.  In  other  words,  using  the  cut  scores  that  defined 
the  quintiles  in  the  sample  of  experienced  recruiters,  we  calculated  how  many  students  in  the 
ARC  would  have  scored  in  each  category  on  the  overall  composite.  Figure  3  shows  that  the 
majority  of  the  students  would  have  scored  in  the  highest  quintile  for  experienced  recruiters. 
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Given  the  validity  of  the  NSAB  composite  shown  in  Figure  2,  these  results  suggest  that  many  of 
the  ARC  students  would  perfonn  well  in  this  job.  However,  a  number  of  individuals  also  scored 
in  the  bottom  quintiles  and,  therefore,  may  not  be  high  perfonners.  In  other  words,  it  appears  that 
the  NSAB  can  be  used  to  identify  high  potential  ARC  students.  Nevertheless,  a  longitudinal 
validation  investigation  is  necessary  to  clearly  examine  the  validity  of  the  NSAB  for  predicting 
the  performance  of  ARC  students  over  time. 
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Figure  2.  Quintile  plots  of  the  relationships  between  the  NSAB  composite  and  Army  commitment,  recruiting  fit,  recruiting  stress,  and  the 
performance  rating  composite. 


Scores  in  the  Recruiting  Student  Sample  (n  =  928) 


NSAB  Composite  Quintiles  from  the  Sample  of  Experienced  Recruiters 


Figure  3.  Percentages  of  ARC  students  in  each  of  the  NSAB  composite  quintiles  identified  in  the 
sample  of  experienced  recruiters. 


Summary 

Our  primary  focus  in  the  recruiting  data  collection  effort  was  to  examine  the  potential  of 
the  NSAB  to  predict  success  in  recruiting.  The  results  demonstrated  that  several  NSAB  scales 
are  significantly  related  to  a  number  of  recruiting  success  criteria.  These  results  are  consistent 
with  past  research  examining  the  predictors  of  recruiter  success  (White  et  ah,  2001)  and  suggest 
that  the  NSAB  may  be  useful  for  selecting  high  potential  individuals  for  recruiting  duty 
assignment. 
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CHAPTER  4:  CONCLUSION 


The  Army  has  conducted  a  series  of  research  activities  to  develop  an  instrument  for  NCO 
duty  assignment.  As  part  of  this  research,  ARI  developed  the  Warrior  Attributes  Inventory 
(WAI)  (White,  Borman,  &  Bowles,  2001).  Predictive  and  concurrent  validation  research  results 
demonstrated  that  the  WAI  was  related  to  recruiter  performance,  production,  and  attrition  from 
recruiter  training.  However,  the  Army  needs  a  new  instrument  that  can  be  more  easily 
administered  to  NCOs  in  an  online,  unproctored  environment.  Delivery  through  an  unproctored, 
web-based  application  would  make  it  far  easier  for  Soldiers  to  complete  the  assessment  in  a 
timely  and  efficient  manner.  TAP  AS,  which  was  specifically  designed  for  computerized 
administration,  provides  a  compelling  alternative  to  the  WAI  for  this  purpose.  Moreover, 

TAPAS  has  the  added  advantage  of  being  resistant  to  faking,  which  has  the  potential  to  increase 
the  validity  of  the  measure  in  operational  settings.  As  such,  TAPAS  was  used  as  the  basis  for  a 
new  instrument,  the  Noncommissioned  Officer  Special  Assignment  Battery  (NSAB). 

The  goal  of  the  current  project  was  to  explore  the  potential  validity  of  the  NSAB  for 
NCO  in-service  testing  and  selection  for  recruiting  duty.  Initial  results  demonstrated  a  significant 
overlap  between  the  NSAB  and  key  dimensions  on  the  WAI  that  had  predicted  recruiter 
performance  in  previous  research.  This,  along  with  the  additional  scales  available  on  the  NSAB 
suggests  that  the  NSAB  is  a  highly  promising  measure  for  predicting  recruiter  perfonnance. 

Our  next  step  was  to  examine  the  validity  of  the  NSAB  with  a  sample  of  experienced 
recruiters  to  more  directly  test  the  relationship  between  the  NSAB  scales  and  recruiter  criteria. 
Results  showed  that  a  composite  of  NSAB  scales  was  useful  for  identifying  high  perfonners. 
Recruiters  in  the  top  20%  on  the  NSAB  composite  were  more  committed  to  the  Anny, 
experienced  less  stress  in  their  job,  reported  a  higher  degree  of  fit  with  recruiting,  and  received 
higher  performance  ratings  than  those  in  the  lowest  scoring  20%.  In  sum,  the  preliminary 
evidence  suggests  that  the  NSAB  can  help  to  identify  Soldiers  with  high  potential  for  recruiting 
duty  success. 

Although  these  results  are  highly  encouraging  from  both  a  theoretical  and  quantitative 
standpoint,  these  efforts  should  be  interpreted  as  a  preliminary  investigation.  Building  on  the 
promising  foundation  of  the  current  project,  future  research  should  evaluate  the  validity  of  the 
NSAB  in  a  longitudinal,  predictive  design,  and  under  operational  conditions.  This  type  of  design 
would  allow  the  Army  to  more  closely  estimate  the  validity  of  an  operational  NSAB. 

The  content  and  features  of  the  NSAB  also  have  potential  for  NCO  selection  for  other 
assignments  such  as  drill  sergeant,  instructors,  or  special  operations.  Previous  research  has 
demonstrated  that  the  NLSI  not  only  predicted  recruiter  success,  but  also  predicted  supervisor 
ratings  of  perfonnance  in  a  sample  of  drill  sergeants  (Kubisiak  et  ah,  2005).  Given  the  close 
conespondence  between  the  constructs  measured  on  the  WAI  (or  NLSI)  and  those  measured  on 
the  NSAB,  and  the  predictive  validity  of  the  NSAB,  the  NSAB  has  great  potential  for  in-service 
testing  for  drill  sergeant  duty  assignments.  Thus,  the  authors  recommend  further  testing  of  the 
NSAB  for  use  as  an  in-service  selection  tool  for  other  NCO  duty  assignments.  As  a  result  of  the 
success  of  the  current  research,  ARI  is  pursuing  implementation  of  the  NSAB  on  an  Army-wide 
accessible  platform  to  support  further  testing  and  evaluation  of  the  NSAB  for  assignments  such 
as  drill  sergeant  or  special  operations. 
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APPENDIX  A.  NCO  ANALYSES 


NCO  Criterion  Measures 


NCO  Criterion  Measures 

Two  sets  of  criterion  measures  were  administered  as  part  of  the  NCO  data  collection:  the 
NCO  Perfonnance  Rating  Scales  and  the  Army  Life  Questionnaire.  These  two  criterion  measures 
are  described  in  more  detail  below. 

NCO  Army-Wide  Performance  Rating  Scales.  The  NCO  Performance  Rating  Scales  were 
developed  by  ARI  to  measure  day-to-day  in-unit  job  perfonnance  across  MOS.  The  scales 
consist  of  fourteen  behaviorally-anchored  rating  scales:  1)  MOS/occupation-specific  knowledge 
and  skill;  2)  common  task  knowledge  and  skill;  3)  communication  skill;  4)  level  of  effort  and 
initiative  on  the  job;  5)  adaptability;  6)  self-management  and  self-directed  learning  skill;  7) 
acting  as  a  role  model;  8)  relating  to  and  supporting  peers;  9)  cultural  tolerance;  10)  leadership 
skills;  1 1)  concern  for  Soldier  quality  of  life;  12)  training  others;  13)  problem  solving/decision 
making  skill;  14)  and  information  management.  Additionally,  there  is  a  scale  measuring  senior 
NCO  potential,  and  an  independent,  single  item  evaluation  of  overall  performance.  These  scales 
were  completed  by  the  NCOs’  supervisors. 

Army  Life  Questionnaire.  The  Army  Life  Questionnaire  (ALQ)  was  completed  by  NCOs  to 
assess  their  attitudes  and  experiences  in  the  Army  (see  Van  Iddekinge,  Putka,  &  Sager,  2005  for 
more  information  on  the  ALQ).  The  ALQ  contains  several  scales  including:  1)  disciplinary 
actions;  2)  affective  commitment;  3)  Anny  fit;  4)  attrition  cognitions;  5)  career  intentions;  6) 
MOS  fit;  7)  MOS  satisfaction;  8)  reenlistment  intentions,  as  well  as  single-item  measures  such  as 
the  Army  Physical  Fitness  Test  (APFT)  score,  and  the  weapons  qualification  score.  There  are 
two  versions  of  the  ALQ,  a  computerized  version  and  a  paper-and-pencil  version  that  was  used  if 
computers  were  not  available.  The  ALQ  item  content  is  identical  across  the  two  versions. 


NCO  Participant  Demographics 

Summary  information  on  key  demographic  characteristics  of  the  participants  is  presented 
in  Tables  A-l  through  A-4. 


A-l 


Table  A-1.  NCO  Gender  and  Ethnicity 


N 

% 

Gender 

Male 

294 

84.5 

Female 

42 

12.1 

Missing 

12 

3.4 

Total 

348 

100.0 

Hispanic,  Latino,  or  Spanish  Origin 

Yes 

52 

15.0 

No 

283 

81.3 

Missing 

13 

3.7 

Total 

348 

100.0 

Race 

American  Indian  or  Alaska  Native 

7 

2.0 

Asian 

9 

2.6 

African-American 

52 

14.9 

Native  Hawaiian/Pacific  Islander 

1 

0.0 

White 

180 

51.7 

2  or  more  selected 

11 

3.2 

Missing 

88 

25.3 

Total 

348 

100.0 

Table  A-2.  NCO  Pay  Grade 

N 

% 

Grade 

E4 

69 

19.8 

E5 

206 

59.2 

E6 

42 

12.1 

E7 

1 

.3 

Missing 

30 

8.6 

Total 

348 

100.0 

Table  A-3.  NCO  Age  and  Time  in  Service 

M 

SD 

Age 

27.69 

5.2 

Time  in  Service  (In  Years) 

6.78 

3.7 

A-2 


Table  A-4.  NCO  MOS 


MOS  Series 

N 

% 

11  -  Infantry 

94 

27.00 

13 -Artillery 

22 

6.32 

19 -Armor 

17 

4.89 

21-  Engineering 

13 

3.74 

25  -  Signal 

39 

11.21 

31  -  Law  Enforcement 

5 

1.44 

42  -  Administrative 

12 

3.45 

68-  Medical 

15 

4.31 

74  -  Chemical  Operations 

14 

4.02 

88 -Transportation 

18 

5.17 

91-  Medical 

30 

8.62 

92  -  Logistics 

32 

9.2 

94-  Electronic  Maintenance 

9 

2.59 

Other 

14 

4.02 

Missing 

14 

4.02 

Total 

348 

100.0 

NCO  Criterion  Measure  Results 
NCO  Performance  Rating  Scales  -  Descriptives 

Each  NCO  was  rated  by  his/her  supervisor  on  fourteen  dimensions  of  perfonnance,  a 
rating  of  senior  NCO  potential,  and  an  independent,  single  item  evaluation  of  overall 
perfonnance.  A  total  of  3 13  supervisor  raters  provided  ratings  for  3 13  NCOs  across  all  nine  data 
collection  sites.  Supervisor  demographics  are  presented  in  Tables  A-5  and  A-6. 


Table  A-5.  Supervisor  Gender  and  Ethnicity 


N 

% 

Gender 

Male 

271 

86.6 

Female 

42 

13.4 

Total 

313 

100.0 

Hispanic,  Latino,  or  Spanish  Origin 

Yes 

51 

16.3 

No 

254 

81.1 

Missing 

8 

2.6 

Total 

313 

100.0 

Race 

American  Indian  or  Alaska  Native 

5 

1.6 

Asian 

6 

1.9 

African-American 

50 

16.0 

Native  Hawaiian/Pacific  Islander 

1 

0.3 

White 

197 

62.9 

2  or  more  selected 

9 

2.9 

Missing 

45 

14.4 

Total 

313 

100.0 
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Table  A-6.  Supervisor  Grade 


N 

% 

Grade 

E5 

29 

9.3 

E6 

74 

23.6 

E7 

172 

55.0 

E8 

30 

9.6 

E9 

1 

0.3 

Missing 

7 

2.2 

Total 

313 

100.0 

As  is  typical  of  performance  ratings,  there  is  a  slightly  negative  skew  to  the  distribution 
of  the  perfonnance  ratings  across  the  7-point  rating  scale  for  the  fourteen  perfonnance 
dimensions,  with  a  lower  percentage  of  ratings  at  the  less  effective  end  of  the  scale.  However, 
while  most  of  the  ratings  fall  in  the  5-6  range,  there  is  reasonable  variability  in  the  ratings, 
suggesting  that  raters  were  differentiating  between  less  effective  and  more  effective  performance 
across  dimensions.  The  means  and  standard  deviations  of  the  perfonnance  ratings  are  provided  in 
Table  A-7  for  each  dimension.  Again,  these  results  show  that  there  was  reasonable  variability  in 
the  ratings. 


A-4 


A- 5 


Table  A-7.  Correlations  Between  the  NCO  Performance  Rating  Scales 


1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

12 

13 

14 

15 

16 

Scales 

1.  MOS  Knowledge  and  Skill 

- 

2.  Common  Task  Knowledge  and  Skill 

.66 

- 

3.  Communication  Skill 

.43 

.55 

- 

4.  Level  of  Effort  and  Initiative 

.54 

.58 

.52 

- 

5.  Adaptability 

.50 

.55 

.49 

.57 

- 

6.  Self-Management  and  Self-  Directed 
Learning  Skill 

.53 

.59 

.55 

.62 

.50 

- 

7.  Acting  as  a  Role  Model 

.44 

.56 

.48 

.55 

.45 

.59 

- 

8.  Relating  to  and  Supporting  Peers 

.48 

.51 

.49 

.50 

.51 

.48 

.55 

- 

9.  Cultural  Tolerance 

.17 

.25 

.18 

.12 

.28 

.19 

.29 

.45 

- 

10.  Leadership  Skills 

.55 

.60 

.54 

.57 

.47 

.59 

.62 

.58 

.28 

- 

11.  Concern  for  Soldier  Quality  of  Life 

.32 

.50 

.47 

.43 

.42 

.45 

.52 

.56 

.41 

.57 

- 

12.  Training  Others 

.61 

.62 

.53 

.60 

.52 

.59 

.59 

.62 

.24 

.66 

.61 

- 

13.  Problem-Solving/Decision-Making 

Skill 

.54 

.60 

.55 

.61 

.51 

.61 

.55 

.59 

.31 

.68 

.57 

.69 

- 

14.  Information  Management 

.52 

.56 

.63 

.50 

.54 

.59 

.44 

.51 

.26 

.60 

.48 

.58 

.64 

- 

15.  Senior  NCO  Potential 

.56 

.63 

.52 

.62 

.52 

.61 

.63 

.57 

.23 

.62 

.47 

.64 

.61 

.60 

- 

Composite 

16.  Performance  Ratings  Composite 

.70 

.78 

.72 

.76 

.71 

.77 

.75 

.76 

.42 

.81 

.70 

.83 

.82 

.76 

.76 

- 

Mean 

5.27 

5.37 

5.01 

5.13 

5.06 

5.05 

5.05 

5.29 

5.63 

5.17 

5.35 

5.10 

5.12 

5.17 

4.95 

5.19 

Standard  Deviation 

1.15 

1.14 

1.29 

1.45 

1.26 

1.25 

1.52 

1.26 

1.05 

1.27 

1.20 

1.35 

1.23 

1.11 

1.52 

.93 

Note.  Bold  values  are  significant  at  the  .05  level.  Sample  sizes  for  these  correlations  ranged  from  288  to  31 3. 


NCO  Performance  Rating  Scales  -  Factor  Analyses 

Correlations  between  the  performance  ratings  are  also  presented  in  Table  A-7.  As  shown, 
a  number  of  these  scales  were  highly  correlated.  Therefore,  we  conducted  factor  analyses  to 
detennine  whether  these  scales  could  be  reasonably  combined  to  create  a  reduced  number  of 
criteria  for  examining  WAI  and  NSAB  validity. 

First,  we  conducted  an  exploratory  factor  analysis  (EFA)  of  the  performance  rating  data. 
The  scree  plot  shown  in  Figure  A-l  indicates  a  very  strong  first  factor.  Therefore,  the  ratings 
were  essentially  unidimensional.  In  addition,  a  confirmatory  factor  analysis  (CFA)  also  indicated 
that  a  single  factor  model  fit  the  data  well  (Root  Mean  Square  Error  of  Approximation  =  .08; 
Comparative  Fit  Index  =  .98;  Standardized  Root  Mean  Square  Residual  =  .05).  The  factor 
loadings  from  this  CFA  model  are  shown  in  Table  A-8.  Based  on  these  results,  we  averaged  the 
15  performance  ratings  into  a  single  variable  and  examined  the  validity  of  the  NSAB  scales  for 
predicting  this  perfonnance  rating  composite.  Table  A-7  shows  the  correlations  between  the 
various  perfonnance  rating  scales  and  the  performance  ratings  composite. 


Figure  A-1.  Scree  Plot  of  the  NCO  Performance  Rating  Scales. 
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Table  A-8.  Factor  Loadings  from  the  Single  Factor  CFA  Model 
of  the  NCO  Performance  Rating  Scales _ 


NCO  Performance  Rating  Scale 

Overall 

Performance 

MOS  Knowledge  and  Skill 

.71 

Common  Task  Knowledge  and  Skill 

.77 

Communication  Skill 

.67 

Level  of  Effort  and  Initiative 

.74 

Adaptability 

.68 

Self-Management  and  Self-  Directed  Learning  Skill 

.75 

Acting  as  a  Role  Model 

.73 

Relating  to  and  Supporting  Peers 

.73 

Cultural  Tolerance 

.39 

Leadership  Skills 

.80 

Concern  for  Soldier  Quality  of  Life 

.66 

Training  Others 

.82 

Problem-Solving/Decision-Making  Skill 

.79 

Information  Management 

.73 

Senior  NCO  Potential 

.80 

A-7 


NCO  Army  Life  Questionnaire  -  Descriptives 

Each  NCO  also  completed  the  ALQ  which  consisted  of  items  measuring  NCO  attitudes 
and  experiences  in  the  Army.  Frequencies  for  several  perfonnance-related  items  are  presented  in 
Table  A-9,  and  the  means,  standard  deviations,  and  correlations  between  the  ALQ  scales  are 
provided  in  Table  A- 10. 

Table  A-9.  Army  Life  Questionnaire  (ALQ)  Performance-related  Item  Frequencies _ 


Variable 

N 

% 

Received  Accelerated  Training  Advancement 

60 

19.4 

Received  Accelerated  Promotion 

151 

46.7 

Received  Accelerated  Position 

169 

52.3 

Completed  Warrior  Leader  Course 

210 

64.0 

Current  Weapon  Qualification: 

Unqualified 

1 

0.3 

Marksman 

56 

17.0 

Sharpshooter 

100 

30.4 

Expert 

172 

52.3 

A-8 
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Table  A-10.  Means,  Standard  Deviations,  and  Correlations  Between  the  ALQ  Scales 


1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

Scales 

1.  APFT  Score 

- 

2.  Promotion  Point  Total* 

-.04 

* 

3.  Army  Fit 

.17 

-.04 

- 

4.  Turnover  Intentions 

-.05 

.08 

-.56 

• 

5.  Affective  Commitment 

.09 

-.01 

.76 

-.48 

- 

6.  Reenlistment  Intentions 

.13 

CXI 

r— 1 

.46 

-.29 

.43 

- 

7.  Career  Intentions 

.05 

.10 

.49 

-.37 

.48 

.78 

- 

8.  MOS  Fit 

.13 

-.06 

.26 

-.18 

.21 

.03 

.08 

| 

9.  Satisfaction  with  MOS 

.10 

-.03 

.47 

-.36 

.44 

.18 

.26 

.62 

- 

Composites 

10.  Army  Commitment 

.12 

.05 

.78 

-.64 

.76 

.82 

.86 

.17 

.40 

- 

11.  MOS  Fit 

.13 

-.05 

.40 

-.30 

.36 

.11 

.18 

.91 

.89 

.31 

- 

Mean 

258.84 

604.73 

3.75 

1.82 

3.39 

3.07 

2.87 

2.98 

3.27 

17.25 

6.25 

Standard  Deviation 

34.10 

162.14 

.75 

.83 

.87 

1.18 

1.38 

1.01 

.91 

3.92 

1.73 

Note.  Bold  values  are  significant  at  the  .05  level.  Sample  sizes  for  these  correlations  range  from  140  to  333;  ‘Promotion  Point  Total  was  computed  using  the  2010 
Promotion  Point  Worksheet  system. 


NCO  Army  Life  Questionnaire  -  Factor  Analyses 

Factor  analyses  were  also  perfonned  on  the  ALQ  scales.  However,  a  distinction  was 
made  between  the  attitudinal  scales  and  the  performance-related  scales  on  the  ALQ.  For 
example,  because  of  their  importance  for  the  Anny  and  their  conceptual  differences  from  the 
other  ALQ  scales,  Army  Physical  Fitness  Test  (APFT)  scores  and  promotion  points  were 
evaluated  separately  from  the  rest  of  the  ALQ  scales.  Table  A- 10  shows  that  these  scales  were 
not  highly  correlated  with  the  other  ALQ  scales,  suggesting  that  treating  these  scales  separately 
was  appropriate.  Consequently,  the  factor  analyses  were  only  conducted  on  the  ALQ  scales 
assessing  Army  fit,  turnover  intentions,  affective  commitment,  reenlistment  intentions,  career 
intentions,  MOS  fit,  and  MOS  satisfaction. 

The  scree  plot  for  these  scales  is  illustrated  in  Figure  A-2.  As  shown,  these  scales  were 
not  unidimensional.  In  fact,  the  scree  plot  suggested  that  a  two-factor  model  would  fit  the  data 
better.  Therefore,  we  conducted  a  two-factor  CFA  on  the  ALQ  scales  with  Anny  fit,  turnover 
intentions,  affective  commitment,  reenlistment  intentions,  and  career  intentions  loading  on  the 
first  factor  and  MOS  fit  and  MOS  satisfaction  loading  on  the  second  factor.  In  addition,  because 
of  their  similar  content,  the  error  terms  for  the  career  intentions  and  reenlistment  intentions 
scales  were  allowed  to  correlate.  The  factor  loadings  from  this  model  are  provided  in  Table 
A-l  1.  We  labeled  the  two  factors  in  the  CFA  model  Anny  Commitment  and  MOS  Fit  and  the 
resulting  model  fit  the  data  well  (Root  Mean  Squared  Error  of  Approximation  =  .03; 
Comparative  Fit  Index  =  1.00;  Standardized  Root  Mean  Squared  Residual  =  .03).  Table  A-10 
provides  the  means,  standard  deviations,  and  conelations  between  these  broad  ALQ  factors  and 
the  original  ALQ  scales.  Given  the  factor  analytic  results,  we  examined  relationships  among  the 
NSAB  scales,  the  performance  rating  composite,  Army  Commitment,  MOS  Fit,  and  APFT 
scores. 


Scree  Plot 
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Table  A-11.  Completely  Standardized  Factor  Loadings  from  the 
2-Factor  CFA  Model  of  the  ALQ  Scales  in  the  NCO  Sample 


ALQ  Army 
Commitment 

ALQ  MOS  Fit 

ALQ  Army  Fit 

.91 

ALQ  Turnover  Intentions 

-.61 

ALQ  Affective  Commitment 

.84 

ALQ  Reenlistment  Intentions 

.50 

ALQ  Career  Intentions 

.55 

ALQ  MOS  Fit 

.56 

ALQ  Satisfaction  with  MOS 

1.10 

NSAB  Validity  Analyses:  NCO  Sample 

Table  A- 12  shows  the  correlations  of  the  NSAB  Scales  with  the  performance  rating 
composite  and  the  ALQ  criteria.  For  comparison,  correlations  with  the  WAI  Recruiter  Selection 
Composite  are  also  included.  As  shown,  a  number  of  NSAB  facets  were  significantly  related  to 
the  criteria.  For  example,  the  NSAB  Achievement  scale  was  correlated  with  Anny  Commitment 
and  APFT  scores  (r  =  .22  and  r  =.18,  respectively). The  NSAB  Physical  Conditioning  scale  was 
highly  correlated,  r  =  .27,  with  APFT  scores.  In  addition,  a  number  of  the  NSAB-criterion 
correlations  were  larger  than  the  correlations  involving  the  current  composite.  In  particular,  the 
current  composite  was  not  a  strong  correlate  of  Anny  Commitment  (r  =  .05).  However,  several 
NSAB  scales  (e.g.,  Achievement,  Physical  Conditioning,  Non-Delinquency,  and  Optimism)  were 
significant  correlates  of  this  criterion.  Therefore,  it  appears  that  the  NSAB  captures  content  that 
is  similar  to  and  may  likely  improve  upon,  the  WAI. 

In  addition,  Table  A- 13  shows  the  standardized  regression  weights  from  the  regressions 
of  the  ALQ  criteria  and  the  performance  rating  composite  onto  the  NSAB  scales.  Because  of  the 
small  sample  size,  we  used  a  one-tailed  significance  test  to  identify  predictors.  Results  showed 
significant  predictive  validity  for  the  NSAB  scales.  The  strongest  relationships  were  with  the 
Anny  Commitment  and  MOS  Fit  variables.  For  these  criteria,  the  multiple  R’s  were  .41  and  .37 
and  several  NSAB  scales  were  significant  predictors.  In  addition,  the  multiple  R  for  APFT  scores 
was  .35,  indicating  that  this  criterion  was  also  predicted  well  by  the  NSAB.  In  contrast,  the 
perfonnance  rating  composite  was  not  predicted  well  by  the  NSAB  scales.  Specifically,  only  the 
Achievement  scale  was  significantly  related  to  this  criterion  and  Table  A- 12  indicates  that  the 
WAI  Recruiter  Selection  Composite  was  also  unconelated  with  these  ratings.  Thus,  the 
perfonnance  ratings  are  not  related  to  the  WAI  or  the  NSAB.  There  were  several  limitations  that 
may  have  attenuated  these  relationships.  First,  only  one  supervisor  provided  performance  ratings 
for  each  NCO,  reducing  the  reliability  of  the  ratings.  Second,  rater  training  was  not  provided  for 
several  of  the  NCO  data  collections.  Finally,  the  performance  ratings  may  be  confounded  with 
differences  across  MOS  and,  in  order  to  predict  performance,  it  may  be  necessary  to  standardize 
the  rating  within  occupations.  Unfortunately,  the  sample  sizes  for  most  specialties  were  too  small 
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to  standardize  within  MOS.  The  reason  for  the  lack  of  significant  correlations  with  the 
perfonnance  ratings  is  unclear  and  is  likely  due  to  a  number  of  factors. 


Table  A-12.  Correlations  Between  the  NSAB  Scales  and  Each  NCO  Criterion 


APFT  Scores 

Army 

Commitment 

MOS  Fit 

Performance 

Rating 

Composite 

WAI  Recruiter  Selection  Composite 

.14 

.05 

.14 

.02 

Cooperation 

-.01 

-.02 

-.01 

-.07 

Dominance 

.16 

.14 

.09 

.08 

Even-Tempered 

-.03 

.02 

-.02 

.02 

Attention-Seeking 

.08 

-.10 

-.01 

-.05 

Selflessness 

-.05 

.04 

.06 

-.02 

Achievement 

.18 

.22 

.16 

.12 

Ingenuity 

.04 

.02 

.09 

-.04 

Intellectual  Efficiency 

.05 

.00 

-.05 

.04 

Adjustment 

.02 

-.11 

.14 

.05 

Order 

.05 

.03 

.03 

-.01 

Physical  Conditioning 

.27 

.18 

.15 

.06 

Responsibility 

-.07 

.13 

.03 

.04 

Self-Control 

.03 

.13 

.01 

.06 

Sociability 

.06 

-.01 

-.02 

.00 

Tolerance 

.09 

.05 

-.08 

-.04 

Non-Delinquency 

-.03 

.18 

.12 

.00 

Consideration 

-.05 

-.05 

.02 

-.08 

Optimism 

.10 

.19 

.17 

.08 

Note.  Bold  values  are  significant  at  the  .05  level.  The  sample  sizes  for  these  correlations  range  from  294  to  330. 
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Table  A-13.  Standardized  Regression  Weights  for  the  NSAB  Scales  that  were  Significant 
Predictors  of  Each  NCO  Criterion 


Performance 

Army  Rating 

APFT  Scores  Commitment  MOS  Fit  Composite 

Cooperation 

Dominance 

.13** 

Even-Tempered 

Attention-Seeking 

-.10* 

Selflessness 

.10* 

Achievement 

.16**  .15**  .12* 

Ingenuity 

Intellectual  Efficiency 

Adjustment 

-.13**  .14** 

Order 

Physical  Conditioning 

.22**  .11* 

Responsibility 

-.16** 

Self-Control 

Sociability 

Tolerance 

.10*  -.15** 

Non-Delinquency 

.17**  .15** 

Consideration 

Optimism 

.18**  .14** 

Multiple  R 

.35**  .41**  .37**  .21** 

Adjusted  R 

.27**  .35**  .30**  .00 

Note.  N  =  309-332;  *p  <  .10;  **p  <  .05. 


A-13 


A-14 


APPENDIX  B.  RECRUITING  PILOT  TEST 


Recruiting  Pilot  Test 

Below  we  describe  the  participants  and  data  collection  procedures  for  the  pilot  test  at  the 
Anny  Recruiting  Course,  followed  by  the  pilot  test  results. 

Recruiting  Student  Data  Collection 

Recruiting  Student  Participants 

We  collected  predictor  (NSAB)  data  from  1020  ARC  students  from  May  through  October 
2011.  The  majority  of  the  participants  were  E-5s  (48.8%)  and  E-6s  (44.5%). 

Data  Collection  Procedures  -  Recruiting  Student  Sample 

The  NSAB  was  collected  in  proctored  testing  sessions  during  the  first  week  of  the  ARC. 
Students  accessed  the  web-based  NSAB  through  the  U.S.  Anny  Accessions  Command 
Enterprise  Portal  and  USAREC’s  Headquarters  Support  System  Anny  Recruiting  Course 
Assignment  &  Sponsorship  (ARCAS)  System.  Students  completed  the  NSAB  in  approximately 
30  minutes. 


NSAB  Results  -  Recruiting  Student  Sample 

Table  B-l  provides  the  descriptive  statistics  for  the  NSAB  scales  and  Table  B-2  provides 
the  correlations  between  these  scales.  Prior  to  running  all  analyses,  the  NSAB  data  were  screened 
for  unmotivated  responders.  Respondents  were  flagged  as  potentially  unmotivated  if  their 
observed  response  patterns  contained  an  unusually  low/high  number  of  Statement  1/Statement  2 
selections  (e.g.,  they  tend  to  select  the  first  or  second  statements  in  the  item  pairs  that  are 
administered),  or  their  item/test  response  latencies  were  unusually  fast  (e.g.,  responding  to  items 
in  less  than  1  or  2  seconds). 


B-l 


Table  B-1.  Descriptive  Statistics  for  the  NSAB  Scales  in  the  Student  Sample 


N 

Mean 

Standard 

Deviation 

Min. 

Max 

Cooperation 

928 

-.04 

.41 

-1.71 

1.65 

Dominance 

928 

.23 

.52 

-1.60 

2.87 

Even-Tempered 

928 

.13 

.53 

-1.60 

1.80 

Attention-Seeking 

928 

-.25 

.54 

-1.84 

2.12 

Selflessness 

928 

-.11 

.45 

-1.43 

1.83 

Achievement 

928 

.37 

.61 

-1.67 

2.01 

Ingenuity 

928 

.09 

.49 

-1.71 

1.92 

Intellectual  Efficiency 

928 

.17 

.53 

-1.75 

1.75 

Adjustment 

928 

.17 

.58 

-1.51 

2.17 

Order 

928 

CO 

T— 1 

.52 

-1.86 

1.79 

Physical  Conditioning 

928 

.18 

.59 

-1.75 

2.08 

Responsibility 

928 

.35 

.44 

-1.03 

1.66 

Self-Control 

928 

.09 

.50 

-1.55 

1.59 

Sociability 

928 

.16 

.64 

-1.73 

2.04 

Tolerance 

928 

-.03 

.55 

-1.82 

1.75 

Non-Delinquency 

928 

-.08 

.52 

-2.03 

1.55 

Consideration 

928 

.00 

.51 

-2.04 

1.65 

Optimism 

928 

.40 

.50 

-1.68 

2.28 
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Table  B-2.  Correlations  Between  the  NSAB  Scales  in  the  Student  Sample 


NSAB  Scales 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

12 

13 

14 

15 

16 

17  18 

1.  Cooperation 

- 

2.  Dominance 

.01 

- 

3.  Even-Tempered 

.31 

-.04 

- 

4.  Attention-Seeking 

.07 

.19 

.02 

- 

5.  Selflessness 

.20 

-.01 

.18 

-.02 

- 

6.  Achievement 

.06 

.34 

.02 

.09 

.17 

- 

7.  Ingenuity 

-.01 

.24 

-.01 

.13 

.03 

.19 

- 

8.  Intellectual  Efficiency 

.00 

.31 

.02 

.06 

-.03 

.24 

.34 

- 

9.  Adjustment 

.08 

.15 

.17 

.07 

.02 

-.03 

.09 

.17 

- 

10.  Order 

-.07 

.05 

.00 

-.05 

.08 

.22 

.04 

.07 

-.05 

- 

11.  Physical  Conditioning 

-.05 

.16 

-.02 

.13 

-.04 

.15 

.07 

.10 

.04 

.08 

- 

12.  Responsibility 

.13 

.18 

.07 

-.01 

.14 

.30 

.08 

.11 

.02 

.13 

-.04 

- 

13.  Self-Control 

.08 

.09 

.17 

-.11 

.13 

.23 

.03 

.14 

.06 

.20 

-.03 

.14 

- 

14.  Sociability 

.27 

.25 

.20 

.37 

.15 

.16 

.16 

.09 

.14 

.00 

.10 

.08 

-.02 

- 

15.  Tolerance 

.19 

.03 

.20 

.07 

.32 

.16 

.09 

.04 

.00 

.02 

-.02 

.10 

.11 

.24 

- 

16.  Non-Delinquency 

.10 

.05 

.14 

-.07 

.18 

.16 

-.07 

.00 

.00 

.10 

-.02 

.13 

.19 

.05 

.08 

- 

17.  Consideration 

.24 

.05 

.21 

.08 

.21 

.10 

.12 

.06 

.02 

.04 

-.04 

.11 

.11 

.28 

.18 

.10 

- 

18.  Optimism 

.16 

.18 

.23 

.16 

.11 

.14 

.06 

.08 

.26 

-.01 

.07 

.09 

.06 

.29 

.09 

.04 

.13 

Note.  Bold  values  are  significant  at  the  .05  level.  The  sample  size  for  these  correlations  was  928. 
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APPENDIX  C.  ARMY  RECRUITER  PERFORMANCE  RATING  SCALES 


Arm/  Recruiter  Performance  Rating  Scales 


A  Locating  And  Contacting  Qualified  Prospects 


How  effective  is  each  Recruiter  in  contacting  large  numbers  of  personslikely  to  join  theArmy;  skillfully  using  available  recruiting 
aidsto  gain  theattention  and  interest  of  young  peopleeligiblefor  Army  service;  knowing  where  and  when  to  prospect;  persisting 
in  prospecting  and  following  up  on  leads? 


Low 

Moderately  Low 

Moderately  High 

High 

Exerts  little  effort 

prospecting;  for  example, 
often  fails  to  follow  up  on 
even  promising  leads,  and 
uses  recruiting  tools  (e.g., 
telephone,  RWS)  sparingly 
and  ineffectively. 

Exerts  effort  prospecti  ng,  but 
may  use  a  limited  number  of 
recruiting  tools  and  may  not 
spend  enough  ti  me  with  or 

direct  sufficient  effort  toward 

the  most  productive  sources 
and  prospects 

Uses  a  number  of  sources  and 

recruiting  toolsfor 
prospecti  ng  and  is  effective  at 
locating  and  contacting 
qualified  prospects. 

D  i  spl  ays  excepti  onal 
ingenuity  and  energy  and 
uses  a  w i  de  vari  ety  of 
recruiting  tools  very 
effectively  to  locate  and 
contact  qualified  prospects. 

O  © 

©  0  © 

©  ©  © 

©  © 
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B.  Gaining  And  Maintaining  Rapport 


How  effective  is  each  Recruiter  in  being  hospitable  to  prospects  and  influencers;  establishing  rapport  with  and  gaining  thetrust, 
and  respect  of  prospectsand  influencers;  adjusting  to  applicants'  styles  and  acting  appropriately  with  different  types  of  applicants 
and  influencers? 


Low 

Moderately  Low 

Moderately  High 

High 

1  s  very  poor  at  gai  ni  ng  and 

Has  trouble  interacting  with 

1  s  typi  cal  1  y  abl  e  to  put 

1  nteracts  very  effectively  with 

mai  ntai  ni  ng  rapport;  appears 

certai  n  prospects;  someti  mes 

prospects  at  ease,  and 

all  types  of  prospects;  is 

disinterested  in  applicants  or 

appears  di  si  nterested  i  n  a 

mai  ntai  ns  good  rapport  with 

excel  1  ent  at  gai  ni  ng  and 

may  answer  questions  in  an 

prospect  or  may  have  a 

them;  interacts  with  most 

mai  ntai  ni  ng  rapport  and 

impersonal  way. 

standard  approach  to 
interacting  that  is 
i  nappropri  ate  for  some 
prospects. 

prospects  i  n  a  warm  and 
fri  endly  way. 

establ  ishi  ng  trust  with 
prospects. 

O  © 

©  o  © 

©  ©  © 
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C.  Obtaining  Information  From  And  About  Prospects  And  Making  Good  Person-Army  Fits 


How  effective  is  each  Recruiter  in  demonstrating  good  listening  and  interviewing  skills;  making  accurate  judgments  and 
suggesting  optionsto  match  prospects’  needsand  preferences;  effectively  obtaining  information  about  prospects  from  other 
sources  (e.g.,  high  school  principal,  parents)  to  assess  their  qualifications  and  needs? 


Low 

Moderately  Low 

Moderately  High 

High 

Is  very  poor  at  getting 

Sometimes  fails  to  learn 

Is  good  at  blueprinting  most 

Always  blueprints  effectively, 

prospects  to  reveal  their  goals 

enough  about  prospects  to 

prospects,  evaluating  their 

identifying  prospects’  goals, 

and  passions,  making  it 

identify  their  primary  goals 

goals,  interests,  and  passions, 

interests,  and  career 

difficult  to  suggest 

and  passions;  may  suggest 

and  then  discussing  Army 

motivations  and  then  is 

appropriate  Army 

Army  features  and  benefits 

opportunities  appropriate  for 

excellent  at  emphasizing 

opportunities;  may 
misinterpret  prospect’s 
interests  and  suggest  features 
or  programs  that  clearly  don’t 
interest  prospect. 

that  do  not  result  in  a  good 

match  with  the  individual’s 

goals,  interests  or  needs. 

meeting  those  needs. 

Army  features  and 
opportunities  that  address 

these  needs  and  motives. 

O  © 

©  o  © 

©  ©  © 
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D.  Counsel  i  ng/Mentori  ng  Ski  I  Is 


How  effective  is  each  Recruiter  in  skillfully  persuading  prospects  to  join  theArmy;  selecting  and  adapting  counseling  techniques 
appropriate  to  different  prospects,  effectively  presenting  Army  benefits  and  opportunities;  identifying  and  overcoming  obstacles 
to  joining  theArmy;  persisting  to  engender  commitment? 


Low 

Moderately  Low 

Moderately  High 

High 

Fails  to  present  Army 

features/  benefitsto  influence 

individual  prospects,  and  is 
frequently  unable  to  identify 

or  overcome  obstacles  to 

joining  theArmy;  often  misses 
opportunities  to  engender 

commitment  even  when  it’s 

clearly  appropriate  to  do  so. 

Presents  Army  features  and 
benefits  in  away  that  is 

sometimes  not  suitable  for  an 

individual  prospect  and  may 
not  identify  obstacles 
preventing  prospects  from 
joining  theArmy;  at  times, 
misses  opportunities  to 
engender  commitment. 

Presents  Army  features/ 
benefits  so  that  most  prospects 

become  more  interested  in  the 

Army;  recognizes  and  is 
prepared  to  overcome 
frequently  encountered 
obstacles  to  joining  theArmy; 

knows  when  and  how  to 

engender  commitment  in  most 

situations. 

Presents  Army  life  and 
benefits  in  a  highly 
appropriate  and  convincing 
way  for  each  prospect,  and  is 
very  adept  at  identifying  and 
overcoming  any  obstacles  to 
joining  theArmy;  rarely 
misses  opportunities  to 
engender  commitment. 

o  @€>0600®©  © 
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E.  Future  Soldier  Training  Program  (FSTP) 


How  effective  is  each  Recruiter  in  skillfully  relaying  accurate  information  about  BCT,  Army  life,  and  theArmy’s  expectations  so 
that  prospects/  recruits  know  what  to  expect;  training  Future  Soldiers  (FSs)  to  standards  on  pre-BCT  tasks;  following  up  and 
maintaining  relationships  with  FSs  and  their  influences,  effectively  overcoming  obstacles  and  concerns  to  ensure  FSs  maintain 
enlistment  intentions? 


Low 

Moderately  Low 

Moderately  High 

High 

Failsto  maintain  contact  with 

FSs  after  they  enlist;  provides 

FSswith  minimal  or 

inaccurate  information  about 

Army  life;  rarely  prepares 
them  for  BCT,  and  exerts  little 

effort  to  counsel  individuals 

who  no  longer  wish  to  be  in 
the  Army. 

Sporadically  contacts  FSs  after 
mandatory  follow-up;  may 
miss  signs  of  concerns  or 
apathy;  spends  little  time 
training  FSs  to  standard  on 
pre-BCT  tasks,  and  may 
provide  incomplete 
information  about  Army  life. 

Followsup  with  FSs  and  their 
influencers  as  needed; 
responds  to  concerns  in  a 
sensitive  manner;  worksto 

train  FSs  to  standard  and 

accurately  describesArmy 
expectations  and  BCT  to 
prepare  them  for  Army  life. 

Maintains  contact  with  and 

provides  emotional  support  to 
all  FSs  and  their  influencers; 
thoroughly  prepares  FSs  for 

BCT  and  Army  life  by  training 

them  to  standard  and 

providing  complete,  detailed 
information  about  Army  life 
and  expectations. 

o  ©  ©00©  0  ©  ©  © 
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F.  Establishing  And  Maintaining  Good  Relationships  In  The  Community 


How  effective  is  each  Recruiter  in  contacting  and  working  effectively  with  individualsand  agencies  capable  of  helping  with 
prospects;  presenting  a  good  imageand  building  a  good  reputation  for  the  Army  by  developing  positive  relationships  with 
persons  in  the  community;  presenting  a  good  Army  image  through  appearance,  language,  and  demeanor? 


Low 

Moderately  Low 

Moderately  High 

High 

Avoids  personal  contact  or 

alienates  individuals  in  the 

community  by  making 
demands  or  failing  to  honor 
commitments;  presents 
negative  image  of  the  Army  by 
poor  personal  appearance  or 

behavior. 

O  © 

Does  not  make  regular  contact 
with  community  agencies  that 
might  be  helpful  in  recruiting, 
and  does  not  develop 
rel ationshi ps  fu  1  ly;  is  not 
particularly  alert  to 
opportunities  to  promotethe 
Army. 

©  o  © 

Spends  productive  time  with 
individuals/  agencies, 
maintaining  a  good  image, 
and  keeping  them  informed  of 
most  Army  activities;  may 
arrangeArmy  activitiesfor 
community  persons  who  can 
help  in  recruiting. 

©  ©  © 

Is  exceptionally  alert  to  and 
adept  at  developing 
relationships  with  relevant 
individualsand  community 
agencies,  and  promotes  a 
strongly  positive  image  of  the 
Army;  may  volunteer  off-duty 
timeto  help  in  the  community. 

©  © 
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G.  Organizing  Skills/Processing  Skills/Time  Management 


How  effective  is  each  Recruiter  in  planning  ahead;  organizing  time  efficiently;  completing  paperwork  accurately  and  on  time; 
keeping  track  of  appointments;  knowing  how  to  locate  and  interpret  important  reference  materials  (e.g.  qualifications,  regulations, 
standards);  ensuring  that  recruits  are  processed  by  quickly  and  efficiently  getting  them  to  the  M  EPS  and  into  the  Army? 


Low 

Moderately  Low 

Moderately  High 

High 

Consistently  fails  to  complete 
necessary  forms;  or  may  use 
wrong  forms;  organizes  time 
poorly  and  does  not  maintain 
Recruiter  Zone;  fails  to  locate 

or  use  reference  materials. 

Som  et  i  m  es  com  p  1  etes 
paperwork  late,  occasionally, 
with  significant  errors;  is 

somewhat  inefficient  in  use  of 

time  and  may  at  times 
schedule  appointments 
without  considering  other 
events;  may  misinterpret 

reference  materials. 

Usually  completes  paperwork 
on  time  and  with  few  errors; 
keeps  Recruiter  Zone  up-to- 
date;  and  generally  uses  time 
efficiently;  typically  utilizes 
reference  materials  correctly. 

Accurately  completes  all 
paperwork,  prior  to  or  as 
scheduled;  devises  plans  so  as 

to  achieve  own  and  station 

goals;  maintains  up-to-date 

and  accurate  Recruiter  Zone 

and  scheduleswork  activities 

very  efficiently  and  effectively; 
consistently  locates  and 

utilizes  reference  materials 

accurately. 

O  © 
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H.  Supporting  Other  Recruiters  And  USAREC 


How  effective  is  each  Recruiter  in  coordinating  activitieswith  and  supporting  other  recruitersto  maximize  the  productivity  of  the 
station  and  battalion;  complying  with  and  supporting  orders  and  directives  from  higher  levels;  mentoring  or  providing  helpful 
tips  and  constructive  feed  back  to  other  Army  recruiters,  especially  if  they  are  new? 


Low 

Moderately  Low 

Moderately  High 

High 

Rarely  cooperates  with, 

May  assist  other  recruiters 

Supports  the  Command  in 

Is  always  enthusiastic  and 

supports,  or  helps  other 

when  specifically  asked  but 

ways  that  are  helpful;  usually 

worksto  build  group  spirit; 

recruiters,  even  if  requested, 

does  not  look  for  opportunities 

places  station/  battalion 

consistently  helps  other 

and  lets  others  carry  the 

to  help  or  support  others; 

mission  above  personal  goals, 

recruiters,  even  when  he/  she 

recruiting  load;  may  interfere 

often  complains  about 

and  generally  cooperates  and 

is  busy;  always  accurately 

with  the  group  effort  by 

mission,  or  having  to  work 

workswell  with  fellow 

shares  information  so  as  to 

withholding  important 

information  or  not 

coordinating  own  activities 

with  others. 

extra  time. 

recruiters. 

increase  group  production. 

O  © 

©  o  © 

©  ©  © 

©  © 
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I.  Overall  Effectiveness 


How  effectively  does  each  Recruiter  perform  overall? 


Low 

Moderately  Low 

Moderately  High 

High 

Performs  below  standards  for 

Usually  achieves  and 

Consistently  achieves  and 

Exceed  s  stan  d  ard  s  an  d 

recruiters. 

maintains  the  standards 

maintains  the  standards 

expectations  for  recruiters. 

expected  of  a  recruiter,  but 

expected  of  a  recruiter,  and 

sometimes  falls  short  on 

som  et  i  m  es  exceed  s  st  an  d  ard  s 

important  areas  of 
performance. 

in  some  areas  of  performance. 

o 

© 

©  o  © 

©  ©  © 

©  © 
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APPENDIX  D.  RECRUITING  LIFE  QUESTIONNAIRE 


There  are  two  sections  in  this  survey.  Please  read  the  instructions  in  each  section  carefully  before 
you  begin. 

Section  One 

Using  the  scale  below,  please  indicate  the  extent  to  which  you  agree  with  the  following  statements. 


Strongly  Agree 

Agree 

Neither  Agree/Disagree 

Disagree 

Strongly  Disagree 

Not  applicable 

A 

B 

C 

D 

E 

F 

Working  Environment  &  Job  Satisfaction 

1 .  I  am  satisfied  with  recruiting  duty. 

2.  I  am  proud  to  be  a  representative  of  the  Army  as  a  recruiter. 

3.  The  stress  in  this  recruiting  job  is  high. 

4.  Recruiting  duty  is  worse  than  I  expected  before  I  received  my  duty  assignment. 

5.  I  feel  like  1  am  part  of  the  Army  “family”. 

6.  1  am  pressured  to  continue  recruiting  even  after  the  station  meets  the  monthly  mission. 

7.  I  enjoy  being  a  recruiter. 

8.  Being  a  recruiter  is  a  big  part  of  who  1  am. 

9.  1  have  recruited  less  than  two  prospects  into  the  Army. 

10.  1  often  feel  burned  out  at  work. 

11.  1  feel  a  strong  sense  of  belonging  to  the  Army. 

12.  Recruiting  is  a  good  match  for  me. 

13.  I  feel  a  sense  of  accomplishment  when  1  enlist  a  recruit. 

14.  1  enjoy  the  constant  pressure  to  recruit. 

15.  Being  a  recruiter  has  a  great  deal  of  personal  meaning  to  me. 

16.  1  have  the  necessary  abilities  to  succeed  in  recruiting. 

17.  The  Army  has  a  great  deal  of  personal  meaning  to  me. 

18.  The  workload  in  recruiting  is  exhausting. 

Personal  &  Professional  Development 

19.  The  training  1  received  in  the  Army  Recruiter  Course  was  helpful  and  relevant  for  my  job  as  a 
recruiter. 

20.  1  have  NOT  attended  any  NCOES  schools  (e.g.,  Warrior  Leader  Course). 

21.  1  have  access  to  training  opportunities  that  will  help  improve  my  recruiting  skills. 

22.  Working  as  a  recruiter  helps  me  achieve  my  long  term  career  goals. 

23.  My  time  in  recruiting  will  increase  my  chances  for  promotion. 

24.  As  a  recruiter  1  have  learned  new  skills  that  will  make  me  a  better  NCO. 

25.  1  have  time  to  engage  in  activities  that  enrich  me  personally. 
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Section  Two 


The  following  items  are  general  questions  about  your  experiences  as  a  recruiter.  Please 
click  next  to  the  statement  that  best  describes  your  response  to  each  question. 

26.  If  you  volunteered  for  recruiting,  what  were  your  reasons  for  becoming  a  recruiter?  (check  all  that 
apply) 

a)  N/A,  1  did  not  volunteer  for  recruiting  duty 

b)  I  was  able  to  choose  the  location  of  my  duty  station 

c)  1  wanted  a  change  from  my  military  specialty  /  occupation 

d)  Recruiting  duty  is  career  enhancing 

e)  Recruiting  duty  is  necessary  for  promotion 

f)  1  believe  in  my  Service  and  want  to  share  it  with  others 

g)  1  want  to  help  young  people 

h)  I  was  ready  for  a  challenge 

i)  1  had  no  choice 

j)  I  wanted  a  change  from  being  deployed 

k)  Other: _ 

27.  On  average,  what  are  the  total  number  of  hours  per  day  you  spend  performing  recruiting-related 
duties,  not  including  PT? 

a)  5  or  fewer  hours 

b)  6-7  hours 

c)  8-9  hours 

d)  10-11  hours 

e)  12  or  more  hours 

28.  Compared  to  other  Army  recruiters  in  your  Company,  would  you  say  you  are: 

a)  One  of  the  best  (exceed  95  percentile) 

b)  Better  than  most  (66  to  95  percentile) 

c)  Average  (35  to  65  percentile) 

d)  Below  average  (below  35  percentile) 

29.  1  have  been  assigned  to  recruiting  duty  for  less  than  2  months. 

a)  Yes 

b)  No 

30.  Do  you  plan  to  reclassify  your  MOS  to  79R? 

a)  Yes 

b)  No 

c)  Undecided 

d)  Already  have  reclassified 

31.  If  you  had  the  freedom  to  select  an  assignment  next  month,  which  of  the  following  would  you 
choose? 

a)  Remain  in  recruiting 

b)  Return  to  my  previous  military  specialty/occupation 

c)  Select  a  totally  new  military  specialty/occupation 

d)  Leave  the  Service 
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APPENDIX  E.  EXPERIENCED  RECRUITER  DEMOGRAPHICS 


Table  E-1.  Recruiter  Gender,  Ethnicity,  and  Marital  Status 


N 

% 

Gender 

Male 

796 

94.0 

Female 

51 

6.0 

Missing 

7 

... 

Total 

854 

100 

Hispanic  Declaration 

Yes 

717 

84.7 

No 

130 

15.3 

Missing 

7 

... 

Total 

854 

100 

Race 

American  Indian  or  Alaska  Native 

9 

1.1 

Asian 

17 

2.0 

African-American 

160 

18.9 

Native  Hawaiian/Pacific  Islander 

7 

0.8 

White 

632 

74.7 

More  than  2  selected 

21 

2.5 

Missing 

8 

... 

Total 

854 

100 

Marital  Status 

Married 

633 

74.3 

Separated 

22 

2.6 

Divorced 

106 

12.4 

Widowed 

1 

0.1 

Never  Married 

90 

10.6 

Missing 

2 

... 

Total 

854 

100 
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Table  E-2.  Recruiter  Grade,  Component,  and  Education 


N 

% 

Grade 

E5 

210 

25.0 

E6 

493 

58.8 

E7 

134 

16.0 

E8 

2 

0.2 

Missing 

15 

... 

Total 

854 

100 

Component 

Regular  Army 

843 

98.9 

Army  National  Guard  or  Reserve 

9 

1.1 

Missing 

2 

... 

Total 

854 

100 

Education 

Less  than  12  years  of  school 

1 

0.1 

High  School  Certificate  or  GED 

28 

3.3 

High  School  Diploma 

86 

10.1 

Some  college,  but  did  not  graduate 

518 

60.9 

Associate’s  Degree 

150 

17.6 

Bachelor’s  Degree 

54 

6.3 

Some  Graduate  School 

9 

1.1 

Graduate  Degree 

5 

0.6 

Missing 

3 

... 

Total 

854 

100 

Table  E-3.  Recruiter  Age  and 

Tenure  (All  in  Years) 

M 

SD 

Age 

31.71 

4.91 

Time  in  Service 

11.26 

3.77 

Time  in  Grade 

3.34 

2.06 

Time  in  Recruiting 

2.34 

.56 

Time  in  Recruiting  Station 

2.26 

.55 
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APPENDIX  F.  RECRUITER  SUPERVISOR  AND  PEER  RATER  DEMOGRAPHICS 


Table  F-1.  Rater  Gender,  Ethnicity,  and  Marital  Status 


N 

% 

Gender 

Male 

593 

94.1 

Female 

37 

5.9 

Missing 

257 

... 

Total 

887 

100 

Hispanic  Declaration 

Yes 

142 

16.1 

No 

740 

83.9 

Missing 

5 

... 

Total 

887 

100 

Race 

American  Indian  or  Alaska  Native 

15 

1.7 

Asian 

16 

1.9 

African-American 

136 

15.8 

Native  Hawaiian/Pacific  Islander 

10 

1.2 

White 

686 

79.4 

Missing 

24 

... 

Total 

887 

100 

Marital  Status 

Married 

691 

77.9 

Separated 

29 

3.3 

Divorced 

102 

11.5 

Widowed 

3 

0.3 

Never  Married 

62 

7.0 

Missing 

0 

... 

Total 

887 

100 

Table  F-2.  Rater  Education 

N 

% 

Less  than  12  years  of  school 

1 

0.0 

High  School  Certificate  or  GED 

12 

1.4 

High  School  Diploma 

79 

8.9 

Some  college,  but  did  not  graduate 

487 

54.9 

Associate’s  Degree 

155 

17.5 

Bachelor’s  Degree 

103 

11.6 

Some  Graduate  School 

36 

4.1 

Graduate  Degree 

14 

1.6 

Missing 

0 

... 

Total 

887 

100 
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Table  F-3.  Rater  Grade 


N 

% 

E5 

115 

13.1 

E6 

346 

39.4 

E7 

355 

40.4 

E8 

62 

7.1 

E9 

1 

0.0 

Missing 

8 

... 

Total 

887 

100 

Table  F-4.  Rater  Role/Position 

N 

% 

Recruiter 

561 

63.3 

Station  Commander 

215 

24.2 

Other 

111 

12.5 

Missing 

0 

... 

Total 

887 

100 

Table  F-5.  Rater  Time  in  Grade  and  Service 

M 

SD 

Time  in  Grade  (In  Years)  3.54 

Time  in  Service  (In  Years)  13.78 

2.28 

5.17 
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